Commit Graph

632 Commits

Author SHA1 Message Date
Adrià Casajús 44e0dd8635
Break using an alias as a mailbox loop in the email_handler.py (#1624)
* Do not allow to use email alias as account email when linking

* Add missing status

* Remove TODO

* Also break contact as email loop

* Better test names

* Allow a reverse alias to send an email to an alias

* Ident fix

* Removed invalid test

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-03-13 13:01:00 +01:00
Adrià Casajús 3fcb37f246
Reformat base64 encoded messages to shorter lines (#1575)
* Reformat base64 encoded messages to shorter lines

* Remove storing debug versions

* Add  example test email

* Update linelength to 76

* Revert changes in pre-commit

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-02-21 15:28:06 +01:00
Adrià Casajús 0a197313ea
Fix: allow receive email from non-canonical sources (#1545)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-01-25 13:17:20 +01:00
Adrià Casajús 81eb56e213
Tranfer aliases to a new mailbox when deleting mailboxes (#1534)
* Set up npm clean install instead of npm install in order to keep the version of npm packages 🎨

* Add option to transfer the alias to a new mailbox when a mailbox is deleted

* Moved alias transfer to job

* Lint

* Update forms

* Revert dockerfile change

Co-authored-by: ewen <ewen.coppens@a1.digital>
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-01-17 11:55:34 +01:00
Adrià Casajús 9378b8a17d
Fix: Return email in the get_communication_email always and search for the alias when needed (#1497)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-12-19 09:23:53 +01:00
Adrià Casajús 5e48d86efa
Canonicalize emails from google and proton before registering users (#1493)
* Revert "Revert "Use canonical email when registering users (#1458)" (#1474)"

This reverts commit c8ab1c747e.

* Only canonicalize gmail and proton

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-12-14 11:50:36 +01:00
Adrià Casajús 73c0429cad
Fix: Set oneclick link for unsubscribe of the newsletter for tx emails (#1465)
* Feat: Add unsub oneclick to the base transactional email template

* Format

* Removed unused

* Format

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-12-13 16:59:14 +01:00
Adrià Casajús c8ab1c747e
Revert "Use canonical email when registering users (#1458)" (#1474)
* Revert "Use canonical email when registering users (#1458)"

This reverts commit f728b0175a.

* missing chang

* typo
2022-12-08 10:57:46 +01:00
Adrià Casajús f728b0175a
Use canonical email when registering users (#1458)
* Use canonical email for registration, check both when checking if user exists

* Fix test

* Set pagesize to 100

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-11-30 17:19:55 +01:00
Spitfireap b849d1cfa7
Simpler csv export (#1383)
* Export alias in csv

* reformating

* template

* Improved contributing script and doc

* Updated test

* removed csv export from GDPR export archive

* added test for new route

* fix trailing space

* moved test to new utils file
2022-11-23 13:51:08 +01:00
Son Nguyen Kim 989358af34
Fix empty authorized address (#1423)
* not allow empty authorized address

* check authorized address before adding

* use github for flake8

* fix test
2022-11-15 16:04:31 +01:00
Adrià Casajús 4661972f97
Fix: When re-sending emails if they trigger exceptions move out of failed dir (#1411)
* Fix: When re-sending emails if they trigger exceptions move out of failed dir

* Use proper timeout

* Lint

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-11-10 13:24:46 +01:00
Adrià Casajús dace2b1233
Fix: Do not re-re-deliver unsent mails on failure to re-deliver (#1397)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-11-03 17:48:09 +01:00
Adrià Casajús 90d60217a4
Feat: Re-deliver mails (#1394)
* Feat: Send undelivered emails

* Add cron job

* Added to the crontab

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-11-02 15:51:14 +01:00
Adrià Casajús 6d8fba0320
Added too many exceptions test (#1378)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-27 14:04:03 +02:00
Adrià Casajús f6463a5adc
Change: Do not sleep on exclusive zones (#1375)
* Change: Do not sleep on exclusive zones

* Update test

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-27 10:40:33 +02:00
Adrià Casajús 7f9ce5641f
Feat: Added parallel limiter to prevent sqlalchemy transaction issues (#1360)
* Feat: Added parallel limiter to prevent sqlalchemy transaction issues

* Remove logs

* Moved initialization to its own file

* Throw exception

* Added test

* Add redis to gh actions

* Added v6 to the name

* Removed debug prints

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-27 10:07:02 +02:00
Adrià Casajús d324e2fa79
Fix: Add csrf verification to directory updates (#1358)
* Fix: Add csrf verification to directory updates

* Update templates/dashboard/directory.html

* Added csrf for delete account form

* Fix tests

* Added CSRF check for settings page

* Added csrf to batch import

* Added CSRF to alias dashboard and alias transfer

* Added csrf to contact manager

* Added csrf to mailbox

* Added csrf for mailbox detail

* Added csrf to domain detail

* Lint

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-27 10:04:47 +02:00
Son Nguyen Kim c71824c68e
Init daily metric (#1351)
* Add DailyMetric model

* increment nb_new_web_non_proton_user

* fix test

* fix test
2022-10-14 17:35:34 +02:00
Son Nguyen Kim 1fc75203f2
Improve test: disable rate limit during test and avoid conflicts between tests (#1349)
* disable rate limit during test, avoid conflict between tests

* fix test
2022-10-14 16:37:49 +02:00
Son Nguyen Kim 7b24cdd98a
Revert "remove deduct_limit as it has no effect (#1347)" (#1348)
This reverts commit 851ba0a99a.
2022-10-13 22:00:45 +02:00
Son Nguyen Kim 851ba0a99a
remove deduct_limit as it has no effect (#1347)
* remove deduct_limit as it has no effect

- disable rate limit during test
- randomize data in test
- support non-empty db in test

* fix more test
2022-10-13 18:55:22 +02:00
Son Nguyen Kim 1c5a547cd0
do not quarantine an email if fails DMARC but has a small rspamd score (#1337)
* do not quarantine an email if fails DMARC but has a small rspamd score

* use 0 when cannot parse rspamd score

* use -1 as default value
2022-10-10 10:13:07 +02:00
Son Nguyen Kim 5088604bb8
Replace reverse alias (#1335)
* replace any reverse alias by real address for all contacts

* improve logging

* fix comment

* Request contacts in batches of 100 to avoid loading the db

* Fix typo

* Added tests for the contact replacement

* Increase batch size to 1k

* Revert and use only reply_email and website_email

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-10 10:00:19 +02:00
Adrià Casajús faeddc365c
Display recovery codes for mfa only once (#1317)
* Recovery codes can only be shown after adding a 2FA code and cannot be seen afterwards

* Added recovery codes fix

* Updated models and script

* Formatting

* Format

* Added base code

* Updated wording

* Set the config by default

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-10-03 12:32:45 +02:00
Adrià Casajús faaff7e9b9
Handle failed payments subscriptions in paddle (#1327)
* Handle failed payments subscriptions in paddle

* Added tests

* Remove unused import

* Remove unused import

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-09-30 17:51:06 +02:00
Son Nguyen Kim 753a28e886
handle case msg is string in replace() (#1271)
should fix https://sentry.io/organizations/simplelogin/issues/3563106404/?alert_rule_id=2478639&alert_timestamp=1662404226476&alert_type=email&environment=production&project=1868546&referrer=alert_email
2022-09-07 10:22:11 +02:00
Son Nguyen Kim 192d03fd68
make sure sl_formataddr always return str (#1269) 2022-09-05 15:38:04 +02:00
Son Nguyen Kim 313a928070
Create sl_formataddr to handle unicode for built-in formataddr (#1265)
* Create sl_formataddr to handle unicode for built-in formataddr

* fix circular import
2022-09-05 08:40:24 +02:00
Son Nguyen Kim f69c9583fb
fix proton partner error when self host (#1255)
* fix proton partner error when self host

* fix test

* fix test

* remove a@b.c
2022-09-01 14:59:16 +02:00
Carlos Quintana 7eb44a5947
Fixes for connect with proton on mobile (#1230)
* Fixes for connect with proton on mobile

* Added a test

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-08-12 13:17:21 +02:00
Adrià Casajús 3a75686898
Generate a web session from an api key (#1224)
* Create a token to exchange for a cookie

* Added Route to exchange token for cookie

* add missing migration



Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-08-10 18:48:32 +02:00
Carlos Quintana a9549c11d7
Rate limiting depending on user authenticated status (#1221)
* Rate limiting depending on user authenticated status

* Update app/extensions.py

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>

* Add rate_limiting tests

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>
2022-08-09 14:57:21 +02:00
Carlos Quintana a04152a37f
Do not allow SVG image uploads (#1198) 2022-07-29 08:52:51 +02:00
Adrià Casajús 25fde11a86
Refactor alias suffix (#1194)
* Extract suffix generation and validation to a module

* Updated tests

* Make custom alias use signed suffixes

* Added the signature check to the module

* Fix invalid route

* Move more suffix related stuff

* Fix tests

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-27 17:40:22 +02:00
Adrià Casajús f4c5198055
Remove ResetCodes after email change (#1191)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-26 14:43:31 +02:00
Carlos Quintana 7db3ec246e
Mitigate open redirect with OAuth (#1176)
* Mitigate open redirect with OAuth

* Fix tests
2022-07-21 14:23:08 +02:00
Adrià Casajús 06c1c7f2f7
Restrict the number of free alias for new free users (#1155)
* Restrict the number of free alias for new free users

* Fix test

* Make flag reverse

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-20 11:09:22 +02:00
Adrià Casajús f3d47a1eaa
Allow users to keep the original unsub behaviour (#1148)
* Feature: Preserve original unsubscribe request

* Updated tests

* Updated settings

* PR comments

* reduced prefix length

* Include migrate users for new unsub behaviour

* PR comments

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-19 17:25:21 +02:00
Adrià Casajús 046748c443
Update pre-commit (#1138)
* Update pre-commit

* Upgrade djlint, remove flake8 and add pylint

* Reformat with new djlint version

* Run pre-commit on CI

* Use only python3.10 on CI

* Reformat files with pre-commit

* Run pre-commit against all files

* Reformat

* Added global excludes

* Added pre-commit to the contributing file

* Set python 3.9 as default

* Set language version to python3

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
Co-authored-by: Carlos Quintana <carlos.quintana@proton.ch>
2022-07-04 16:01:04 +02:00
Adrià Casajús 38c9138cdb
Fix: When logging with parter create accounts with lowercase emails (#1137)
* Fix: When logging with parter create accounts with lowercase emails

* Sanitize emails instead of just lowercase them

* linting

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-04 11:51:43 +02:00
Adrià Casajús aabcc8e72a
Feature: Add delete account route for the api (#1132)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-02 16:45:58 +02:00
Adrià Casajús 88dd07e48d
Feature: Use new job status to retry killed jobs (#1130)
* Feature: Use new job status to retry killed jobs

* Set attermpts and time via config

* Update timing condition

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-07-01 11:14:53 +02:00
Adrià Casajús 21feced342
Refactor unsubscribe handling (#1090)
* Refactor unsubscribe email handling

* MR comments

* Moved all unsub logic to the encoder

* remove unused

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-06-30 11:40:01 +02:00
Adrià Casajús c85ed7d29e
Fix: Always treat references header as a string (#1127)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-06-29 19:48:22 +02:00
Carlos Quintana d06470a3c6
Activate users created with account link (#1124) 2022-06-29 16:55:20 +02:00
Carlos Quintana 9abb8aa47f
Validate user uploaded image (#1123)
* Validate user uploaded image

* Fix test/data path detection
2022-06-29 15:04:55 +02:00
Son Nguyen Kim f6a7ee981a
do not send double subscription email (#1118)
* do not send double subscription email

* remove unused import

* remove unused test
2022-06-28 17:51:44 +02:00
Carlos Quintana dd0598a4dd
Send welcome email when user created by login with proton (#1115)
* Send welcome email when user created by login with proton

* Add dedicated test to user.created_by_partner
2022-06-28 11:57:21 +02:00
Carlos Quintana 686f4f3f68
Always check redirect_uri for oauth (#1111)
* Always check redirect_uri for oauth

* Fix OAuth tests
2022-06-27 13:20:18 +02:00
Adrià Casajús de31e6d072
Allow to set sudo mode for api requests (#1108)
* Allow to set sudo mode for api requests

* Rebase migration on top of master

* PR comments

* Added missing migration

* Removed unused import

* Apply suggestions from code review

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-06-23 14:26:36 +02:00
Son Nguyen Kim 09cec0cdec
allow to hide some public domains and set their order (#1107) 2022-06-22 18:21:19 +02:00
Adrià Casajús 99ce10a1bc
Send email to users with a subscription and a partner plan upgrade (#1101)
* Send email to users with a subscription and a partner plan upgrade

* Update double-subscription-partner.html

* Update double-subscription-partner.txt.jinja2

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
Co-authored-by: Son Nguyen Kim <nguyenkims@users.noreply.github.com>
2022-06-20 14:34:20 +02:00
Adrià Casajús fbb59a1531
Send welcome mail to proton created users (#1099)
* Send welcome mail to proton created users

* Skip import

* Use new logo

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-06-20 11:36:16 +02:00
Carlos Quintana 5ee5e386e5
Allow to create users from partner (#1095)
* Allow to create users from partner

* Fix tests

* Update tests/test_account_linking.py

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>

* Fix lint

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>
2022-06-16 10:25:50 +02:00
Carlos Quintana ba6c5f93ac
Add extension_redirect endpoint (#1093)
* Add extension_redirect endpoint

* Add test for extension_redirect
2022-06-16 09:56:00 +02:00
Carlos Quintana 332fcb27d9
Fix double backslash open redirect (#1096) 2022-06-16 09:55:08 +02:00
Carlos Quintana 56ec95bc93
Fix proton integration issues (#1071)
* Fix proton integration issues

* Make external_user_id non nullable

* Fix tests
2022-06-10 16:21:56 +02:00
Carlos Quintana c0fe10def6
Raise proper exception on account already linked error (#1069)
* Raise proper exception on account already linked error

* Update app/account_linking.py

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>

* Fix FMT

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>
2022-06-10 12:23:04 +02:00
Carlos Quintana c0a4c44e94
Separate code for proton callback handler (#1040)
* Separate code for proton callback handler

* Upgrade migration

* Use simple_login endpoint from Proton API

* Remove unused classes

* Rename Dto class to Data

* Push rename

* Moved link to PartnerUser to allow subscriptions to depend only on it

* Fix test

* PR comments

* Add unique user_id constraint to PartnerUser

* Added more logs

* Added more logs

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-06-09 10:19:49 +02:00
Adrià Casajús e688f04d6b
Send full user report asynchronously on request (#1029)
* Send full user report asynchronously

* Fix test

* Filter some fields before exporting

* Fix: Domain -> CustomDomain

* format settings html

* not include RefusedEmail as they are not usable by user and are automatically deleted

* send the export to the user email

* change email and setting wording

* fix user can only export data once

* remove alias export section

* remove unused import

* fix flake8

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
Co-authored-by: Son <nguyenkims@users.noreply.github.com>
2022-06-07 10:45:04 +02:00
Carlos Quintana dba56f0dae
Store hmaced partner api tokens (#1028)
* Store hmaced partner api tokens

* MR comments
2022-06-02 11:24:04 +02:00
Adrià Casajús 7ba9bcb9e2
Save unsent emails to disk to be resent later (#1022)
* Initial save to disk

* Store unsent messages to disk so they can be retried later

* Set back not sending emails

* Fixed decorator

* Add general exceptions to the catchall

* Have dummy server just to make sure

* Added several server test cases

* ADded tests for bounced and error status

* Moved dir creation to config parse time

* Set LOG.e

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-05-30 11:52:10 +02:00
Carlos Quintana 5ab943e12c
Remove get_proton_partner_id function 2022-05-23 16:43:06 +02:00
Carlos Quintana ed9d2ed816
Receive partner as param in ProtonCallbackHandler 2022-05-23 16:11:58 +02:00
Son 9e8a419994 add test 2022-05-20 18:12:53 +02:00
Adrià Casajús 220f21bb2a
Fix tests to allow re-running them locally without colliding with previous runs 2022-05-20 14:39:07 +02:00
Adrià Casajús 3330625426
Allow mailer to keep in mem mails to validate tests 2022-05-19 12:27:06 +02:00
Adrià Casajús 2adcbf52be
Merge pull request #963 from simple-login/ac-complaints
Handle complaints that have multiple recipients
2022-05-16 10:30:14 +02:00
Adrià Casajús 64c67f4429
PR comments 2022-05-13 18:14:21 +02:00
Adrià Casajús 34ad81c7c0
Merge pull request #921 from simple-login/ac-free-no-reverse-alias
Prevent free users from creating reverse-alias
2022-05-13 17:13:48 +02:00
Adrià Casajús 9066116b7e
Simplified method 2022-05-12 18:33:13 +02:00
Adrià Casajús 4d07bc9d31
Moved global flag to config 2022-05-12 18:30:46 +02:00
Adrià Casajús caff70ea38
Set global config to enable/disable feature 2022-05-12 16:35:51 +02:00
Adrià Casajús 5dde39eb37
Prevent free users from creating reverse-alias 2022-05-12 13:20:05 +02:00
Adrià Casajús 2660c96fa7
Fix: Track processes that start with the same chars independently (smtp vs stmpd) (#974)
* Fix: Track processes that start with the same chars independently (smtp vs stmpd)

* Format

* Added more test

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2022-05-12 12:37:19 +02:00
Son d2fad44003 create a constant for paddle grace days 2022-05-11 19:03:27 +02:00
Son 2573c68e82 use a grace period of 14 days for paddle subscription 2022-05-11 17:24:52 +02:00
Carlos Quintana e09d7a2b71
Fix open redirect 2022-05-11 14:50:37 +02:00
Adrià Casajús 48554369bd
Get the mailbox if possible from the email log 2022-05-10 23:34:57 +02:00
Adrià Casajús 6c13f7de05
refactored to reduce duplicated codepaths 2022-05-10 18:23:14 +02:00
Adrià Casajús a2f141d3cc
Get recipient address from the complaint report when possible 2022-05-10 17:54:51 +02:00
Son 21255866b6 improve test 2022-05-10 14:53:50 +02:00
Son 4dbbc4ed5e add test 2022-05-10 11:23:40 +02:00
Carlos Quintana 8d4683e59e
Add login with proton 2022-05-05 12:20:55 +02:00
Adrià Casajús eea436875a
Merge pull request #914 from simple-login/ac-store-contact-bounces
Store bounces in the reply phase to prevent abuse
2022-05-03 14:44:07 +02:00
Adrià Casajús 56159765d9
Rename 2022-05-02 11:53:32 +02:00
Son 2de5161cd2 run alembic in run-test.sh 2022-05-01 17:38:14 +02:00
Son c748ab22e6 run db migration in github ci 2022-05-01 17:06:10 +02:00
Adrià Casajús e62022f032
Merge remote-tracking branch 'origin/master' into ac-store-contact-bounces
* origin/master: (29 commits)
  PR comments
  support "enabled" param in /api/v2/aliases
  Update PGPy to 0.5.4 to allow for python 3.10
  Also install libpq-dev
  Fix python 3.10
  Add methods to check if alias will be auto-created
  PR comments
  Allow sending messages in a background thread
  Use the proper import for newrelic agent
  not send emails to inform about an alias can't be created to disabled user
  prevent disabled user from using the api
  make sure disabled user can't create new alias
  Put version version between " so it is 3.10 instead of 3.1
  Add workflow for python 3.10
  Remove it for all creds
  Do not send the transports to the js part since we have not stored them previously
  move help to menu on small screen
  only show the help button on desktop
  use another logo for mobile
  add new parameter disabled in /GET /api/v2/aliases
  ...
2022-04-29 15:56:09 +02:00
Adrià Casajús 04399e827e
Merge pull request #940 from simple-login/ac-check-auto-create
Add methods to check if an alias will be auto-created
2022-04-29 12:05:18 +02:00
Son a2c477a816 support "enabled" param in /api/v2/aliases 2022-04-28 17:24:35 +02:00
Adrià Casajús 8e35a09788
Add methods to check if alias will be auto-created 2022-04-28 15:10:38 +02:00
Adrià Casajús 9a04376894
Allow sending messages in a background thread 2022-04-28 14:43:24 +02:00
Son 7b7cb0b571 prevent disabled user from using the api 2022-04-27 16:24:38 +02:00
Son eab7606f93 make sure disabled user can't create new alias 2022-04-27 16:06:54 +02:00
Adrià Casajús 5208c549fa
Rename TransactionalComplaint to ProviderComplaint 2022-04-25 14:40:42 +02:00
Son 58b332b7bc add new parameter disabled in /GET /api/v2/aliases 2022-04-25 09:22:29 +02:00
Adrià Casajús c573ef655e
Store bounces in the reply phase to prevent abuse 2022-04-21 11:23:58 +02:00
Son b61670fbc0 remove unused import 2022-04-21 09:26:44 +02:00