Commit graph

651 commits

Author SHA1 Message Date
Son Nguyen Kim
493a5daa45 use warning level 2021-09-08 18:25:40 +02:00
Son Nguyen Kim
ff2cbeb3af handle case to header isn't present 2021-09-08 15:49:47 +02:00
Son Nguyen Kim
a58cf9dd5e use warning level for log 2021-09-08 15:23:48 +02:00
Son Nguyen Kim
5ac78f2694 reformat 2021-09-08 11:29:55 +02:00
Son Nguyen Kim
a6e8684afb add more debug info 2021-09-08 11:25:53 +02:00
Son Nguyen Kim
201eb3b9b9 fix handle_hotmail_complaint: handle email format with name 2021-09-08 11:02:35 +02:00
Son Nguyen Kim
b53da25a41 handle hotmail complaint 2021-09-06 19:44:18 +02:00
Son Nguyen Kim
fb29503b81 do not send bounce to IgnoreBounceSender 2021-08-02 11:33:58 +02:00
Son Nguyen Kim
67cd7ae3d4 keep Reply-To header, replace it by a reverse-alias 2021-07-28 09:12:52 +02:00
Son Nguyen Kim
d706bbbd4b Log "Custom/email_handler_time" to NewRelic 2021-07-23 15:48:50 +02:00
Son Nguyen Kim
7c65d92cc1 better logging 2021-07-13 09:59:59 +02:00
Son Nguyen Kim
f97b18e60a fill up EmailLog.alias_id when creating new EmailLog 2021-07-11 12:28:42 +02:00
Son Nguyen Kim
7ae60b9d82 alert phishing attempt 2021-07-11 08:40:10 +02:00
Son NK
6532e0de93 Return 550 instead of 421 when rate limited. Rename greylisting to rate limit 2021-06-24 09:47:01 +02:00
Son NK
6d67c02311 refactor 2021-06-23 19:57:21 +02:00
Son NK
e8cee6de80 increase greylisting threshold 2021-06-23 19:55:41 +02:00
Son NK
70b51b5002 return 550 instead of 421 in case of SMTPRecipientsRefused 2021-06-23 19:50:42 +02:00
Son NK
945496f67d use warning for ignore email 2021-06-23 19:47:51 +02:00
Son NK
6fa267e92b refactor: put all SMTP statuses into status.py 2021-06-23 19:47:06 +02:00
Son NK
58a1d6e783 add warning for when postfix queue id can't be retrieved 2021-06-23 18:19:13 +02:00
Son NK
b84eb13ab5 Discard ignored email
Create IgnoredEmail model
2021-06-22 17:52:24 +02:00
Son NK
00e0b69c76 accept email sent to an alias from its mailbox 2021-06-04 17:45:51 +02:00
Son NK
79d0ef8906 Use Postfix queue-id as log message-id 2021-06-04 17:15:59 +02:00
Son NK
d53796c8d9 use warning level for icloud bounce handling 2021-06-04 15:23:48 +02:00
Son NK
dd591c7437 fix 2021-06-02 16:27:48 +02:00
Son NK
54f806fc4d handle icloud bounce 2021-06-02 11:46:00 +02:00
Son NK
3897f6b633 refactor handle_bounce() 2021-06-02 11:38:52 +02:00
Son NK
09abdffda3 remove msg logging for auto reply case 2021-05-30 20:02:41 +02:00
Son NK
809a50f7d1 Handle out-of-office email during forward phase 2021-05-30 19:58:08 +02:00
Son NK
1750ad45d5 fix message logging 2021-05-28 17:46:52 +02:00
Son NK
60a1f48e6e take into account BOUNCE_PREFIX_FOR_REPLY_PHASE when handling bounces 2021-05-25 17:59:40 +02:00
Son NK
73555ad524 generate mail_from during reply phase using BOUNCE_PREFIX_FOR_REPLY_PHASE 2021-05-25 17:59:40 +02:00
Son NK
3c6c3f7dbd add log when creating a new EmailLog 2021-05-24 12:08:30 +02:00
Son NK
ce22e16285 add logging for case reverse alias receiving email from <> 2021-05-22 17:11:32 +02:00
Son NK
078f3e8188 Set the "X-SimpleLogin-Envelope-To" to the alias during forward 2021-05-12 10:46:07 +02:00
Son NK
ecdef797f9 generate a message_id at the beginning of email processing 2021-05-06 17:20:33 +02:00
Son NK
c003dd0b01 create a copy msg for every recipient except the last one 2021-05-06 17:08:30 +02:00
Son NK
1f4637c064 add logging for message id 2021-03-29 10:27:19 +02:00
Son NK
365c11f926 only run spam check in email handler if ENABLE_SPAM_ASSASSIN is enabled 2021-03-26 10:00:48 +01:00
Son NK
66eb93fe53 fix sanitize header 2021-03-17 10:59:13 +01:00
Son NK
aadf2e1939 reusing the msg already sanitized 2021-03-17 10:23:35 +01:00
Son NK
5cba2eaa38 sanitize header 2021-03-17 10:23:35 +01:00
Son NK
826e4455cf refactor 2021-03-17 10:23:35 +01:00
Son NK
d0dd64bf7b change method order 2021-03-17 10:23:35 +01:00
Son NK
11789559f1 move spf_pass(), sl_sendmail() to email_utils.py 2021-03-17 10:23:35 +01:00
Son NK
d1d81e6a6d move get_spam_score_async(), get_spam_score() to email/spam.py 2021-03-17 10:23:35 +01:00
Son NK
9a1dc0240b improve logging 2021-03-16 09:17:23 +01:00
Son NK
7ec0405709 improve message_id 2021-03-16 09:15:59 +01:00
Son NK
2d7219c218 flake8 2021-03-15 20:00:10 +01:00
Son NK
82154ec858 ignore email sent from a reverse-alias 2021-03-15 19:55:22 +01:00
Son NK
7811f06fc1 remove asyncio_main() 2021-03-15 19:49:14 +01:00
Son NK
6eb7ebc338 add message_id in log to keep track of an email processing 2021-03-15 19:41:42 +01:00
Son NK
157b7adbda improve logging 2021-03-08 12:11:47 +01:00
Son NK
26613cdeeb fix handling auto reply 2021-03-08 12:09:27 +01:00
Son NK
796ad58dca improve logging 2021-03-08 12:08:23 +01:00
Son NK
31ff8b962b handle the auto responder email 2021-03-06 17:44:46 +01:00
Son
6f37bf858d take into account NOT_SEND_EMAIL in sl_sendmail 2021-03-06 16:23:19 +01:00
Son
3f7d325e6e remove the exception logging when a transaction isn't found 2021-02-06 16:00:32 +01:00
Son NK
00fde00d53 black 2021-01-28 18:10:29 +01:00
Son NK
2831cd04d8 delete all headers in forward phase 2021-01-28 17:26:02 +01:00
Son NK
40e4d8e232 Use POSTFIX_PORT_FORWARD during forward phase 2021-01-28 13:50:24 +01:00
Son NK
3544db8f1c sanitize contact.website_email in bounce 2021-01-26 10:04:03 +01:00
Son NK
4cd49b66c2 use VERP for transactional email: remove SENDER, SENDER_DIR 2021-01-26 09:59:22 +01:00
Son NK
3e1ef3358b Create bounce when handling bounce 2021-01-26 09:59:22 +01:00
Son NK
4a91db8e11 rename parse_email_log_id_from_bounce -> parse_id_from_bounce 2021-01-26 09:59:22 +01:00
Son
fcc04ba929 handle case where email_log is deleted 2021-01-25 21:27:34 +01:00
Son NK
0de5b5a9bf revert change 2021-01-25 18:45:38 +01:00
Son NK
604ba285b1 replace "550 SL E3" by "550 5.1.1 SL E3 " 2021-01-25 17:34:41 +01:00
Son NK
0d6338b525 fix disable alias email subject 2021-01-19 10:36:11 +01:00
Son NK
f94b82c134 remove unused import 2021-01-19 10:33:20 +01:00
Son NK
d1d7a93ca5 remove handle_bounce_deprecated 2021-01-19 10:27:57 +01:00
Son NK
7986ff0819 Handle the case msg.get_all return Header object (and not string) 2021-01-15 11:30:43 +01:00
Son NK
f4218a0693 Log exception when handle_bounce_deprecated is used 2021-01-13 11:03:44 +01:00
Son NK
c431abd917 take into account alias.cannot_be_disabled in should_disable 2021-01-13 11:03:30 +01:00
Son NK
56864ff0df improve log 2021-01-11 15:53:08 +01:00
Son NK
8764a050d5 fix orig email not correctly uploaded in handle_bounce_forward_phase() 2021-01-11 15:25:54 +01:00
Son NK
a044c47295 fix 2021-01-11 15:10:46 +01:00
Son NK
f1e9b2b5d7 use VERP: send email from bounce address 2021-01-11 14:55:55 +01:00
Son NK
01858ac452 sanitize contact email 2021-01-11 12:27:02 +01:00
Son NK
ef7fae32b1 remove the "Hi {name}" from email template 2021-01-11 10:23:34 +01:00
Son NK
7e53b97f81 handle the case where alias is deleted in handle_bounce_reply_phase() 2021-01-04 19:25:15 +01:00
Son NK
30593f9c78 store spam report 2021-01-04 14:43:57 +01:00
Son NK
aae63006c6 handle bounce report sent to alias 2021-01-04 14:22:07 +01:00
Son NK
291b9a7d55 use warning level for reply message detected as spam 2020-12-31 11:30:53 +01:00
Son NK
01da9aafcd retry get_spam_score 1 more time 2020-12-31 11:26:12 +01:00
Son NK
1c22e14f68 SMTPServerDisconnected can also happen when creating SMTP server object 2020-12-31 11:22:45 +01:00
Son NK
0df4d1a93d add more logging 2020-12-29 12:17:24 +01:00
Son NK
e06f3dc209 fix template 2020-12-21 11:57:12 +01:00
Son NK
6441c22bcd use warning level for email and mailbox same domain error. Update email wording. 2020-12-21 09:39:26 +01:00
Son NK
da5bb6f9b5 no need to add alias to To header 2020-12-19 17:28:18 +01:00
Son NK
e48f19afb5 use info level for set message-id 2020-12-18 15:34:01 +01:00
Son NK
efc6b32ce0 black 2020-12-16 20:34:31 +01:00
Son NK
866ef1c139 increase the max_nb_alert 2020-12-16 20:30:27 +01:00
Son NK
2f803e4714 refactoring: move template to folder 2020-12-16 18:51:14 +01:00
Son NK
ae9abe8512 remove unused var 2020-12-16 18:50:09 +01:00
Son NK
30b2182694 use should_disable to decide whether an alias should be disabled 2020-12-16 18:50:09 +01:00
Son NK
8d72d66d08 keep References and In-Reply-To in reply phase for a correct email thread 2020-12-11 11:13:19 +01:00
Son NK
b2e1682704 do not override message-id in forward phase 2020-12-11 11:12:38 +01:00
Son NK
c1ad161db7 add email_log to get_spam_score 2020-12-11 11:05:01 +01:00
Son NK
93503d4cd3 Do not rely on revert to delete EmailLog object when pgp fails 2020-12-11 11:03:52 +01:00
Son NK
f03bde1d8d remove _MESSAGE_ID 2020-12-11 11:02:52 +01:00
Son Nguyen Kim
bf139f83b3
Merge pull request #342 from herrboyer/linting
Linting
2020-12-07 17:45:50 +01:00
Renaud Boyer
5d948faf56 black 2020-12-06 22:11:58 +01:00
Son NK
ce2d68a64d take into account include_sender_in_reverse_alias when creating reverse-alias 2020-12-06 19:37:20 +01:00
Renaud Boyer
b16bfaac35 liniting 2020-12-06 13:54:59 +01:00
Son NK
d150dfacdb rename contact_from_header -> from_header 2020-12-05 18:15:53 +01:00
Son NK
8c5f311367 prefer using Reply-To header when creating a new contact 2020-12-05 18:15:00 +01:00
Son NK
b7fdbe7721 log more 2020-12-02 12:40:29 +01:00
Son NK
4d89ac4158 replace replace_str_in_msg() by replace() 2020-11-30 15:15:44 +01:00
Son NK
22b082fd55 use warning for invalid contact email 2020-11-30 10:50:16 +01:00
Son NK
59745b68d0 use warning level when cannot encrypt using python-gnupg 2020-11-27 20:35:57 +01:00
Son NK
e7b83fadbc trigger build 2020-11-26 10:38:48 +01:00
Son NK
3efa96020b use warning level for invalid contact email 2020-11-26 10:06:16 +01:00
Son NK
e2f0a72ab7 log headers in case a contact email is skipped 2020-11-26 09:49:03 +01:00
Son NK
dcd116f11a use mail_from as fallback when contact_email is not valid 2020-11-25 17:50:25 +01:00
Son NK
ac1a6f5613 skip invalid contact in replace_header_when_forward() 2020-11-25 15:21:01 +01:00
Son NK
dfcb74dc87 do not create contact with invalid email in get_or_create_contact() 2020-11-25 15:20:42 +01:00
Son NK
609d59d23f improve logging 2020-11-25 15:20:00 +01:00
Son NK
2f882b81fe sleep before retry 2020-11-25 14:56:54 +01:00
Son NK
5e1a68cdee retry 1 more if SMTPServerDisconnected 2020-11-25 14:43:02 +01:00
Son NK
4f211bba61 fix subject not correctly decoded in spf-fail email 2020-11-25 09:53:27 +01:00
Son NK
272c5628bb fix 2020-11-24 17:02:09 +01:00
Son NK
297857a140 Remove _MAILBOX_ID_HEADER 2020-11-24 16:50:55 +01:00
Son NK
5231483026 add doc 2020-11-24 16:38:54 +01:00
Son NK
fb465ba03e use exception log for the case where mailbox is an alias 2020-11-24 16:38:49 +01:00
Son NK
54942cdf65 set EmailLog.mailbox_id in forward and reply phase. A EmailLog for each mailbox in forward phase. 2020-11-24 16:38:34 +01:00
Son NK
c28872288b black 2020-11-24 11:28:14 +01:00
Son NK
c707342695 Use pgp_enabled() instead of pgp_finger_print 2020-11-24 11:22:41 +01:00
Son NK
bcdf522174 create normalize_reply_email(): handle case where reply email contains space, quote, etc 2020-11-22 13:07:09 +01:00
Son NK
f069d2f083 use getaddresses to parse multiple address from To, CC header. Remove get_addrs_from_header() 2020-11-21 19:15:02 +01:00
Son NK
c53fe90484 fix email subject 2020-11-18 16:16:37 +01:00
Son NK
30185a2798 handle the case where reply_email is not ascii 2020-11-18 16:11:00 +01:00
Son NK
78cb49095a fix reply_email not set 2020-11-18 11:48:09 +01:00
Son NK
319078fceb use contact email when generating reply-email 2020-11-18 10:24:39 +01:00
Son NK
9154b4656d refactor: create is_reply_email() 2020-11-16 19:22:19 +01:00
Son NK
75ba1669e0 Create generate_reply_email() and refactor 2020-11-16 19:15:09 +01:00
Son NK
496be08639 handle case where contact address is empty/invalid 2020-11-14 15:55:53 +01:00
Son NK
b4d1b3950d log SA report 2020-11-12 12:11:39 +01:00
Son NK
c3f73b25b2 decode the subject 2020-11-09 21:16:50 +01:00
Son NK
21839d579c log more 2020-11-09 17:03:47 +01:00
Son NK
2cc7cb6a37 use to_bytes instead of .as_bytes() 2020-11-09 17:02:10 +01:00
Son NK
4d03d2fe04 Fix subject 2020-11-07 17:23:28 +01:00
Son NK
78f5f27d5d add more debugging 2020-11-07 16:12:28 +01:00
Son NK
6a68141d8d Use mailbox generic subject for forwarded emails 2020-11-07 13:00:45 +01:00
Son NK
5ef3ab4d74 disable handle_bounce_reply_phase 2020-11-05 10:26:19 +01:00
Son NK
d0ca773376 not forward email that has invalid from address 2020-11-05 10:26:09 +01:00
Son NK
352cd978bd add debug 2020-11-04 19:42:20 +01:00
Son NK
6585aef443 use warning level 2020-11-04 16:11:32 +01:00
Son NK
3dee121bec improve handle_bounce_reply_phase 2020-11-04 15:38:26 +01:00
Son NK
3a03dec077 simplify code 2020-11-04 14:55:54 +01:00
Son NK
009236e623 add debug code when bounce message cannot be parsed 2020-11-04 13:37:33 +01:00
Son NK
6c626520d3 handle_bounce_reply_phase 2020-11-04 12:32:15 +01:00
Son NK
5a7df14d58 use msg.as_bytes() to sign instead of as_string() 2020-11-03 13:30:37 +01:00
Son NK
8d0e243c83 sign PGP forwarded email if PGP_SENDER_PRIVATE_KEY 2020-11-02 19:09:57 +01:00
Son NK
63788125da save bounce email sent to an alias when 2020-11-02 15:10:03 +01:00
Son NK
c41c36acaa set "date" header in forward phase if needed 2020-11-02 14:53:22 +01:00
Son NK
38877598cf fix handle_reply: do not delete _MIME_HEADERS headers, add Date header 2020-11-02 14:51:37 +01:00
Son NK
756e8080ab handle case msg.get_payload(decode=True) is None 2020-11-01 18:38:21 +01:00
Son NK
1d0aa0f900 fix doc 2020-11-01 18:13:50 +01:00
Son NK
7337110110 Remove all headers in reply phase 2020-11-01 18:12:09 +01:00
Son NK
593e81705b Handle case Content-Type and Mime-Version are missing in prepare_pgp_message 2020-11-01 18:06:28 +01:00
Son NK
53e57eee42 clone orig message in prepare_pgp_message 2020-11-01 18:06:05 +01:00
Son NK
7ca74eaa6f replace-reverse-alias and pgp encryption before modifying message header in reply phase 2020-11-01 18:02:43 +01:00
Son NK
0307793666 use pgpy as fallback for gpg 2020-10-28 17:07:53 +01:00
Son NK
5a190ed840 use pgpy if python-gnupg fails 2020-10-28 12:21:24 +01:00
Son NK
9210459a72 add pgpy
- add pgpy to poetry
- add test PGP keys to local_data
- add encrypt_file_with_pgpy()
- use randomly pgpy
2020-10-28 11:50:14 +01:00
Son NK
aa1cac521b Set X-SimpleLogin-Envelope-From header in forward phase 2020-10-27 11:03:56 +01:00
Son NK
8f6550f992 update contact.mail_from and contact.from_header if needed 2020-10-27 10:40:54 +01:00
Son NK
4a7b73a218 use warning log for disabled account 2020-10-26 10:33:53 +01:00
Son NK
bdec727cd1 allow mailbox's authorized address to unsubscribe alias 2020-10-23 13:29:20 +02:00
Son NK
bfa59dcdd9 fix 2020-10-22 12:26:45 +02:00
Son NK
ea45ac119e Refactor: create Alias.unsubscribe_link 2020-10-22 10:37:02 +02:00
Son NK
f624085aa3 handle newsletter unsubscribe when the subject=user_id* 2020-10-22 10:34:52 +02:00
Son NK
1fcf166c00 small refactor: add should_add_dkim_signature 2020-10-15 16:24:04 +02:00
Son NK
e79522b638 take into account Premium domains 2020-10-15 16:21:31 +02:00
Son NK
805e78cad1 rename email_belongs_to_alias_domains -> email_belongs_to_default_domains 2020-10-14 18:46:05 +02:00
Son NK
8268568f08 add mailbox.disabled column. Disable a mailbox if it fails tests for 10 days consecutive. 2020-10-12 13:28:21 +02:00
Son NK
6993721ae2 disable email forwards/sending if user is disabled 2020-10-04 12:49:43 +02:00
Son NK
6253a4eb23 set SpamAssassin timeout to 300s 2020-09-30 14:03:19 +02:00
Son NK
abc42df0fb create get_spam_score() as a sync function, use a simpler version for running MailHandler. Remove async/await 2020-09-30 11:05:21 +02:00
Son NK
61e4455406 logging more for spamassassin 2020-09-29 16:00:53 +02:00
Son NK
d7ca639dc1 format 2020-09-29 13:11:04 +02:00
Son NK
65938d2fb7 improve email template 2020-09-29 13:03:15 +02:00
Son NK
e8ccbced59 refactor code: wrap smtp.sendmail into sl_sendmail() 2020-09-29 12:57:14 +02:00
Son NK
17ef292779 update reply-must-use-personal-email template 2020-09-29 11:00:50 +02:00
Son NK
0a7643b367 take into account mailbox authorized address when check for spoofing 2020-09-28 17:43:09 +02:00
Son NK
063885ccf7 Add get_mailbox_from_mail_from() 2020-09-28 17:41:16 +02:00
Son NK
740d31871d remove asyncio.shield 2020-09-17 17:03:20 +02:00
Son NK
5f8fff5af3 wrap shield around aiospamc.check to avoid the task being canceled 2020-09-16 19:47:12 +02:00
Son NK
25118dff9b use parseaddr_unicode instead of parseaddr 2020-09-16 17:28:15 +02:00
Son NK
5c8d31111c add contact to handle_unknown_mailbox logging 2020-09-16 17:24:42 +02:00
Son NK
b9b442294b remove unnecessary sanitize on mailbox email 2020-09-14 20:02:46 +02:00
Son NK
cb2033443c fill up contact mail_from, from_header if possible 2020-09-14 18:22:26 +02:00
Son NK
1ab36bd22b remove unnecessary email address sanitize 2020-09-14 17:38:48 +02:00
Son NK
ed2e748d1e sanitize envelope mail_from and rcpt_tos 2020-09-14 17:30:01 +02:00
Son NK
e6dd2f1717 do not forward to unverified mailbox 2020-09-10 09:38:30 +02:00
Son NK
0c9106717b handle case where contact is concurrently created 2020-09-09 17:00:07 +02:00
Son NK
15d7f6407e fix compatible with mailvelope add name=encrypted.asc 2020-09-08 11:10:22 +02:00
Son NK
e36768824f handle the case contact already added 2020-09-03 19:42:52 +02:00
Son NK
b83c513607 use warning level 2020-09-03 15:43:33 +02:00