Commit graph

677 commits

Author SHA1 Message Date
Son
eeb24f594a Add OLD_UNSUBSCRIBER and support it 2021-11-22 18:17:07 +01:00
Son
dc243d6027 improve logging 2021-11-22 11:23:21 +01:00
Son
77e38e63fe handle hotmail complaint during reply phase 2021-11-21 11:31:28 +01:00
Son
fdfa286d3e allow contact email to be case sensitive 2021-11-18 16:44:04 +01:00
Son
225a3ae750 handle Yahoo complaint for transactional email 2021-11-17 14:37:49 +01:00
Son
2b8de82028 handle hotmail complain for transactional email 2021-11-17 14:32:30 +01:00
Son
790f0ed23c return 250 status when handling bounces 2021-11-13 11:21:19 +01:00
Son
4fc6619553 reduce Hotmail abuse report rate 2021-11-10 10:57:22 +01:00
Son
4c1c8a3dc1 fix msg[headers.MESSAGE_ID] can return str 2021-11-10 09:38:20 +01:00
Son
0e24513bcf fix case where msg[headers.IN_REPLY_TO] can be non str 2021-11-08 11:21:01 +01:00
Son
4214efa497 handle the case original_message_id is None in replace_original_message_id 2021-11-05 09:43:58 +01:00
Son
80bbfb6f4b Parse reverse alias first in handle_hotmail_complaint 2021-11-04 10:40:12 +01:00
Son
334dc01a1b fix url 2021-11-03 10:11:52 +01:00
Son
4369137e25 block the sender via one click unsubscribe 2021-11-02 15:44:43 +01:00
Son
b50f1d60b2 refactor: create headers constants for List-Unsubscribe 2021-11-02 14:36:37 +01:00
Son
52a19818b7 save email whose bounce info can't be parsed for debugging 2021-11-02 14:32:16 +01:00
Son
4a5983993e rename file 2021-11-02 11:46:41 +01:00
Son
cb0d992ecc rename file 2021-11-02 11:37:46 +01:00
Son Nguyen Kim
10be304865
Merge pull request #668 from szepeviktor/patch-2
Remove unused import
2021-11-02 11:02:05 +01:00
Son
f95428a5cc do not delete email log when email can't be sent to a contact 2021-11-02 10:59:17 +01:00
Son
542310f5ca refactor: rename file 2021-11-02 10:58:51 +01:00
Viktor Szépe
271ddb82f2
Remove unused import 2021-11-02 05:39:06 +01:00
Son
6b33e66016 handle the case another matching with original_message_id was created in the mean time 2021-11-01 20:39:53 +01:00
Son
5b5bbcc83c refactor: extract replace_sl_message_id_by_original_message_id 2021-11-01 18:45:10 +01:00
Son
2546fefa51 handle message-id replacement for case a reply is sent to multiple recipients 2021-11-01 18:43:19 +01:00
Son
603e98d0bf remove unnecessary check 2021-11-01 17:58:39 +01:00
Son
cf8150b996 handle case original_message_id can be None 2021-10-28 18:41:36 +02:00
Son
5f6ad21e85 improve bounce email notification 2021-10-28 11:43:44 +02:00
Son
a4dbbb6ac2 if contact is blocked, do not forward email 2021-10-28 10:19:58 +02:00
Son
7c182d20a4 try parsing the alias from the from header which might contain the reverse alias when handling hotmail complaint 2021-10-26 12:16:57 +02:00
Son
a99ac24b72 cron, init app, job runner: wrap in an app context to benefit from app setup like database cleanup, sentry integration, etc 2021-10-26 10:52:28 +02:00
Son
8680c0a739 do do not use the ra+ prefix for reverse alias 2021-10-25 14:33:42 +02:00
Son
de0f838950 ignore hotmail bounce that uses 'Undisclosed recipients:;' in :To header 2021-10-25 10:36:50 +02:00
Son
be7ae3021a rename is_reply_email -> is_reverse_alias, make sure reverse-alias must end with EMAIL_DOMAIN 2021-10-19 12:14:16 +02:00
Son
c5987bcfbb Log message_id 2021-10-19 12:05:41 +02:00
Son
a2fcfbbb20 use as_bytes().decode() instead of as_string() in bounce_info 2021-10-19 12:05:35 +02:00
Son
7952ce7ecf sanity check to make sure the message id hasn't been added before 2021-10-19 11:58:05 +02:00
Son
c12f3b3e7a Use SL message ID during reply phase. Exchange original SL and original message-id during the forward and reply phase to keep email thread. 2021-10-18 17:35:16 +02:00
Son
9c653dbacd no need to fill up message-id header: it seems always filled 2021-10-18 17:34:11 +02:00
Son
d18bb28ca9 add more log to investigate "Cannot parse Postfix queue ID" error 2021-10-15 10:37:22 +02:00
Son
72931aa9b7 fill up Bounce.info 2021-10-14 15:46:52 +02:00
Son
68cf54b2d9 Revert "use async in email handler"
This reverts commit 4d7cd09847.
2021-10-13 10:27:59 +02:00
Son
0ec4a3971c Revert "sleep for 60s when rate limit is hit for the first time"
This reverts commit 2524c8ab98.
2021-10-13 10:27:54 +02:00
Son
becf789d5e Revert "add more logging"
This reverts commit 94a9a1479b.
2021-10-13 10:27:46 +02:00
Son
94a9a1479b add more logging 2021-10-13 10:18:46 +02:00
Son
c132e3fbbc flake8 2021-10-12 15:03:16 +02:00
Son
2524c8ab98 sleep for 60s when rate limit is hit for the first time 2021-10-12 14:53:30 +02:00
Son
4d7cd09847 use async in email handler 2021-10-12 14:48:33 +02:00
Son
eb0e327402 remove "with app.app_context():" 2021-10-12 14:47:01 +02:00
Son
372466ab06 do not use flask-sqlalchemy
- add __tablename__ for all models
- use sa and orm instead of db
- rollback all changes in tests
- remove session in @app.teardown_appcontext
2021-10-12 14:36:47 +02:00
Son
ee0ed7d9ec refactor: use headers.py 2021-10-11 12:21:14 +02:00
Son
d6fc132df1 do not replace message-id in reply phase 2021-10-11 12:13:24 +02:00
Son
5821294ae9 refactor: use headers.py 2021-10-11 12:10:18 +02:00
Son
9bb83fe3e2 fix email thread: do not delete original message id 2021-10-11 12:00:37 +02:00
Son Nguyen Kim
339d611e63 remove Contact.from_header column 2021-09-27 12:19:33 +02:00
Son Nguyen Kim
3ad961bfb9 ignore contact name that has hex ascii code \x00 2021-09-27 10:21:49 +02:00
Son Nguyen Kim
51a7dbfa52 Revert "not dkim sign"
This reverts commit b14534db2c.
2021-09-25 18:47:15 +02:00
Son Nguyen Kim
b14534db2c not dkim sign 2021-09-23 19:32:06 +02:00
Son Nguyen Kim
7f5201effa handle ValueError raised by parse_full_address 2021-09-22 16:39:31 +02:00
Son Nguyen Kim
3b16e502b3 add debug info when an email is sent from reverse-alias 2021-09-22 09:58:40 +02:00
Son Nguyen Kim
16dd35470f add more debug info 2021-09-21 14:20:46 +02:00
Son Nguyen Kim
ba3074b94a use warning instead of error 2021-09-20 16:59:27 +02:00
Son Nguyen Kim
f6fd97ef05 log total number of email log for an alias when it is rate limited 2021-09-20 13:54:29 +02:00
Son Nguyen Kim
17c13ee37f ignore smtp error in handle_bounce_forward_phase() 2021-09-20 13:51:16 +02:00
Son Nguyen Kim
7bb54e1e8e add "Custom/number_incoming_email" in newrelic 2021-09-20 09:48:06 +02:00
Son Nguyen Kim
144418ae47 use debug level when postfix queue id can't be parsed 2021-09-15 09:28:27 +02:00
Son Nguyen Kim
e4ed192cce rename 2021-09-13 19:50:15 +02:00
Son Nguyen Kim
106358da5f handle yahoo complaint 2021-09-13 19:49:40 +02:00
Son Nguyen Kim
b3012376c3 use another DKIM header if one fails 2021-09-13 16:04:32 +02:00
Son Nguyen Kim
6e42e536db ignore email sent from a mailbox to its alias if user.ignore_loop_email 2021-09-10 18:15:22 +02:00
Son Nguyen Kim
f53e8c1af8 refactor 2021-09-10 17:48:36 +02:00
Son Nguyen Kim
61d9f7ee43 refactor 2021-09-10 17:31:29 +02:00
Son Nguyen Kim
638e8137ec fix test 2021-09-10 17:10:05 +02:00
Son Nguyen Kim
3ad4b6b76f use flanker instead of parseaddr_unicode 2021-09-10 17:06:38 +02:00
Son Nguyen Kim
500ff00c7c use flanker to parse To:, CC: header in replace_header_when_forward() 2021-09-10 16:51:36 +02:00
Son Nguyen Kim
6e9dfdd6f1 if the complaint cannot be handled, forward it normally 2021-09-09 18:55:29 +02:00
Son Nguyen Kim
493a5daa45 use warning level 2021-09-08 18:25:40 +02:00
Son Nguyen Kim
ff2cbeb3af handle case to header isn't present 2021-09-08 15:49:47 +02:00
Son Nguyen Kim
a58cf9dd5e use warning level for log 2021-09-08 15:23:48 +02:00
Son Nguyen Kim
5ac78f2694 reformat 2021-09-08 11:29:55 +02:00
Son Nguyen Kim
a6e8684afb add more debug info 2021-09-08 11:25:53 +02:00
Son Nguyen Kim
201eb3b9b9 fix handle_hotmail_complaint: handle email format with name 2021-09-08 11:02:35 +02:00
Son Nguyen Kim
b53da25a41 handle hotmail complaint 2021-09-06 19:44:18 +02:00
Son Nguyen Kim
fb29503b81 do not send bounce to IgnoreBounceSender 2021-08-02 11:33:58 +02:00
Son Nguyen Kim
67cd7ae3d4 keep Reply-To header, replace it by a reverse-alias 2021-07-28 09:12:52 +02:00
Son Nguyen Kim
d706bbbd4b Log "Custom/email_handler_time" to NewRelic 2021-07-23 15:48:50 +02:00
Son Nguyen Kim
7c65d92cc1 better logging 2021-07-13 09:59:59 +02:00
Son Nguyen Kim
f97b18e60a fill up EmailLog.alias_id when creating new EmailLog 2021-07-11 12:28:42 +02:00
Son Nguyen Kim
7ae60b9d82 alert phishing attempt 2021-07-11 08:40:10 +02:00
Son NK
6532e0de93 Return 550 instead of 421 when rate limited. Rename greylisting to rate limit 2021-06-24 09:47:01 +02:00
Son NK
6d67c02311 refactor 2021-06-23 19:57:21 +02:00
Son NK
e8cee6de80 increase greylisting threshold 2021-06-23 19:55:41 +02:00
Son NK
70b51b5002 return 550 instead of 421 in case of SMTPRecipientsRefused 2021-06-23 19:50:42 +02:00
Son NK
945496f67d use warning for ignore email 2021-06-23 19:47:51 +02:00
Son NK
6fa267e92b refactor: put all SMTP statuses into status.py 2021-06-23 19:47:06 +02:00
Son NK
58a1d6e783 add warning for when postfix queue id can't be retrieved 2021-06-23 18:19:13 +02:00
Son NK
b84eb13ab5 Discard ignored email
Create IgnoredEmail model
2021-06-22 17:52:24 +02:00
Son NK
00e0b69c76 accept email sent to an alias from its mailbox 2021-06-04 17:45:51 +02:00
Son NK
79d0ef8906 Use Postfix queue-id as log message-id 2021-06-04 17:15:59 +02:00
Son NK
d53796c8d9 use warning level for icloud bounce handling 2021-06-04 15:23:48 +02:00
Son NK
dd591c7437 fix 2021-06-02 16:27:48 +02:00
Son NK
54f806fc4d handle icloud bounce 2021-06-02 11:46:00 +02:00
Son NK
3897f6b633 refactor handle_bounce() 2021-06-02 11:38:52 +02:00
Son NK
09abdffda3 remove msg logging for auto reply case 2021-05-30 20:02:41 +02:00
Son NK
809a50f7d1 Handle out-of-office email during forward phase 2021-05-30 19:58:08 +02:00
Son NK
1750ad45d5 fix message logging 2021-05-28 17:46:52 +02:00
Son NK
60a1f48e6e take into account BOUNCE_PREFIX_FOR_REPLY_PHASE when handling bounces 2021-05-25 17:59:40 +02:00
Son NK
73555ad524 generate mail_from during reply phase using BOUNCE_PREFIX_FOR_REPLY_PHASE 2021-05-25 17:59:40 +02:00
Son NK
3c6c3f7dbd add log when creating a new EmailLog 2021-05-24 12:08:30 +02:00
Son NK
ce22e16285 add logging for case reverse alias receiving email from <> 2021-05-22 17:11:32 +02:00
Son NK
078f3e8188 Set the "X-SimpleLogin-Envelope-To" to the alias during forward 2021-05-12 10:46:07 +02:00
Son NK
ecdef797f9 generate a message_id at the beginning of email processing 2021-05-06 17:20:33 +02:00
Son NK
c003dd0b01 create a copy msg for every recipient except the last one 2021-05-06 17:08:30 +02:00
Son NK
1f4637c064 add logging for message id 2021-03-29 10:27:19 +02:00
Son NK
365c11f926 only run spam check in email handler if ENABLE_SPAM_ASSASSIN is enabled 2021-03-26 10:00:48 +01:00
Son NK
66eb93fe53 fix sanitize header 2021-03-17 10:59:13 +01:00
Son NK
aadf2e1939 reusing the msg already sanitized 2021-03-17 10:23:35 +01:00
Son NK
5cba2eaa38 sanitize header 2021-03-17 10:23:35 +01:00
Son NK
826e4455cf refactor 2021-03-17 10:23:35 +01:00
Son NK
d0dd64bf7b change method order 2021-03-17 10:23:35 +01:00
Son NK
11789559f1 move spf_pass(), sl_sendmail() to email_utils.py 2021-03-17 10:23:35 +01:00
Son NK
d1d81e6a6d move get_spam_score_async(), get_spam_score() to email/spam.py 2021-03-17 10:23:35 +01:00
Son NK
9a1dc0240b improve logging 2021-03-16 09:17:23 +01:00
Son NK
7ec0405709 improve message_id 2021-03-16 09:15:59 +01:00
Son NK
2d7219c218 flake8 2021-03-15 20:00:10 +01:00
Son NK
82154ec858 ignore email sent from a reverse-alias 2021-03-15 19:55:22 +01:00
Son NK
7811f06fc1 remove asyncio_main() 2021-03-15 19:49:14 +01:00
Son NK
6eb7ebc338 add message_id in log to keep track of an email processing 2021-03-15 19:41:42 +01:00
Son NK
157b7adbda improve logging 2021-03-08 12:11:47 +01:00
Son NK
26613cdeeb fix handling auto reply 2021-03-08 12:09:27 +01:00
Son NK
796ad58dca improve logging 2021-03-08 12:08:23 +01:00
Son NK
31ff8b962b handle the auto responder email 2021-03-06 17:44:46 +01:00
Son
6f37bf858d take into account NOT_SEND_EMAIL in sl_sendmail 2021-03-06 16:23:19 +01:00
Son
3f7d325e6e remove the exception logging when a transaction isn't found 2021-02-06 16:00:32 +01:00
Son NK
00fde00d53 black 2021-01-28 18:10:29 +01:00
Son NK
2831cd04d8 delete all headers in forward phase 2021-01-28 17:26:02 +01:00
Son NK
40e4d8e232 Use POSTFIX_PORT_FORWARD during forward phase 2021-01-28 13:50:24 +01:00
Son NK
3544db8f1c sanitize contact.website_email in bounce 2021-01-26 10:04:03 +01:00
Son NK
4cd49b66c2 use VERP for transactional email: remove SENDER, SENDER_DIR 2021-01-26 09:59:22 +01:00
Son NK
3e1ef3358b Create bounce when handling bounce 2021-01-26 09:59:22 +01:00
Son NK
4a91db8e11 rename parse_email_log_id_from_bounce -> parse_id_from_bounce 2021-01-26 09:59:22 +01:00
Son
fcc04ba929 handle case where email_log is deleted 2021-01-25 21:27:34 +01:00
Son NK
0de5b5a9bf revert change 2021-01-25 18:45:38 +01:00
Son NK
604ba285b1 replace "550 SL E3" by "550 5.1.1 SL E3 " 2021-01-25 17:34:41 +01:00
Son NK
0d6338b525 fix disable alias email subject 2021-01-19 10:36:11 +01:00
Son NK
f94b82c134 remove unused import 2021-01-19 10:33:20 +01:00
Son NK
d1d7a93ca5 remove handle_bounce_deprecated 2021-01-19 10:27:57 +01:00
Son NK
7986ff0819 Handle the case msg.get_all return Header object (and not string) 2021-01-15 11:30:43 +01:00
Son NK
f4218a0693 Log exception when handle_bounce_deprecated is used 2021-01-13 11:03:44 +01:00
Son NK
c431abd917 take into account alias.cannot_be_disabled in should_disable 2021-01-13 11:03:30 +01:00