Commit graph

2759 commits

Author SHA1 Message Date
Carlos Quintana
b931518620
Add create alias list event (#2125)
* chore: add alias create list proto event

* chore: generate python files from proto
2024-06-06 09:05:47 +00:00
Carlos Quintana
6862ed3602
fix: event listener (#2119)
* fix: commit transaction after taking event

* feat: allow to reconnect to postgres for event listener

* chore: log sync events pending to process to metrics

* fix: make dead_letter runner able to process events without needing to have lock on the event

* chore: close Session after reconnect

* refactor: make EventSource emit only events that can be processed
2024-05-24 10:21:19 +02:00
Carlos Quintana
450322fff1
feat: allow to disable event-webhook (#2118) 2024-05-23 16:50:54 +02:00
Carlos Quintana
8eccb05e33
feat: implement HTTP event sink (#2116)
* feat: implement HTTP event sink

* Update events/event_sink.py

---------

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>
2024-05-23 11:32:45 +02:00
Carlos Quintana
3e0b7bb369
Add sync events (#2113)
* feat: add protocol buffers for events

* chore: add EventDispatcher

* chore: add WebhookEvent class

* chore: emit events

* feat: initial version of event listener

* chore: emit user plan change with new timestamp

* feat: emit metrics + add alias status to create event

* chore: add newrelic decorator to functions

* fix: event emitter fixes

* fix: take null end_time into account

* fix: avoid double-commits

* chore: move UserDeleted event to User.delete method

* db: add index to sync_event created_at and taken_time columns

* chore: add index to model
2024-05-23 10:27:08 +02:00
Son Nguyen Kim
b5b167479f
Fix admin loop (#2103)
* mailbox page requires sudo

* fix the loop when non-admin user visits an admin URL

https://github.com/simple-login/app/issues/2101

---------

Co-authored-by: Son NK <son@simplelogin.io>
2024-05-10 18:52:12 +02:00
Adrià Casajús
8f12fabd81
Make hibp rate configurable (#2105) 2024-05-10 18:51:16 +02:00
Daniel Mühlbachler-Pietrzykowski
b6004f3336
feat: use oidc well-known url (#2077) 2024-05-02 16:17:10 +02:00
Adrià Casajús
80c8bc820b
Do not double count AlilasMailboxes with Aliases (#2095)
* Do not double count aliasmailboxes with aliases

* Keep Sl-Queue-id
2024-04-30 16:41:47 +02:00
Son Nguyen Kim
037bc9da36
mailbox page requires sudo (#2094)
Co-authored-by: Son NK <son@simplelogin.io>
2024-04-23 22:25:37 +02:00
Adrià Casajús
015036b499
Prevent proton mailboxes from enabling pgp encryption (#2086) 2024-04-12 15:19:41 +02:00
Son Nguyen Kim
d5df91aab6
Premium user can enable data breach monitoring (#2084)
* add User.enable_data_breach_check column

* user can turn on/off the data breach check

* only run data breach check for user who enables it

* add tips to run tests using a local DB (without docker)

* refactor True check

* trim trailing space

* fix test

* Apply suggestions from code review

Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>

* format

---------

Co-authored-by: Son NK <son@simplelogin.io>
Co-authored-by: Adrià Casajús <acasajus@users.noreply.github.com>
2024-04-12 10:39:23 +02:00
Adrià Casajús
2eb5feaa8f
Small improvements (#2082)
* Update logs with more relevant info for debugging purposes

* Improved logs for alias creation rate-limit

* Reduce sudo time to 120 secs

* log fixes

* Fix missing object to add to the session
2024-04-08 15:05:51 +02:00
Adrià Casajús
3c364da37d
Dmarc fix (#2079)
* Add log to spam check + remove invisible characters on import

* Update log
2024-03-26 11:43:33 +01:00
Adrià Casajús
36cf530ef8
Preserve X-SL-Queue-Id (#2076) 2024-03-22 11:00:06 +01:00
Adrià Casajús
0da1811311
Cleanup old data (#2066)
* Cleanup tasks

* Update

* Added tests

* Create cron job

* Delete old data cron

* Fix import

* import fix

* Added delete + script to disable pgp for proton mboxes
2024-03-18 16:00:21 +01:00
Adrià Casajús
f2fcaa6c60
Cleanup also messsage-id headers from linebreaks (#2067) 2024-03-18 14:27:38 +01:00
Son Nguyen Kim
f5babd9c81
Move import export back to setting (#2063)
* replace black by ruff

* move alias import/export to settings

* fix html closing tag

* add rate limit for alias import & export

---------

Co-authored-by: Son NK <son@simplelogin.io>
2024-03-14 15:56:35 +01:00
Adrià Casajús
e96de79665
Add missing indexes and mark aliases as created by partner (#2058)
* Add missing indexes and mark aliases as created by partner

* Configure if we should skip the partner aliases or not
2024-03-13 14:30:17 +01:00
Daniel Mühlbachler-Pietrzykowski
a608503df6
feat: add generic OIDC connect (#2046) 2024-03-13 14:30:00 +01:00
Adrià Casajús
9719a36dab
Do not replace unsubs that go to UNSUBSCRIBER (#2051) 2024-03-06 16:26:10 +01:00
Adrià Casajús
76423527dd
Update HIBP async script (#2043)
* Update HIBP async script

* Fix: continue instead of return
2024-03-04 13:12:38 +01:00
Adrià Casajús
501b225e40
Require sudo for account changes (#2041)
* Move accounts settings under sudo

* Fixed sudo mode

* Add a log message

* Update test

* Renamed sudo_setting to account_setting

* Moved simple login data export and alias/import export to account settings

* Move account settings to the top-right dropdown
2024-02-29 11:20:29 +01:00
Adrià Casajús
1dada1a4b5
Allow to skip creating transactional emails (#2042) 2024-02-27 16:52:45 +01:00
Adrià Casajús
673e19b287
Sanitize unused next parameter (#2040) 2024-02-26 19:23:03 +01:00
Adrià Casajús
173ae6a221
Allow to soft-delete users (#2034)
* Allow the possibility of soft-deleting users

* Unschedule for delete after link

* Add dry run to the cron
2024-02-22 17:38:34 +01:00
Adrià Casajús
363b851f61
Fix: use proper bucket time for the rate limit 2024-02-20 11:13:06 +01:00
Adrià Casajús
50c130a3a3
Store the latest email_log id in the alias to simplify dashboard query (#2022)
* Store the latest email_log id in the alias to simplify dashboard query

* Fix test

* Add script to migrate users last email_log_id to alias

* Always update the alias last_email_log_id automatically

* Only set the alias_id if it is set

* Fix test with randomization

* Fix notification test

* Also remove explicit set on tests

* Rate limit alias creation to prevent abuse (#2021)

* Rate limit alias creation to prevent abuse

* Limit in secs

* Calculate bucket time

* fix exception

* Tune limits

* Move rate limit config to configuration (#2023)

* Fix dropdown item in header (#2024)

* Add option for admin to stop trial (#2026)

* Fix: if redis is not configured do not enable rate limit (#2027)

* support product IDs for the new Mac app (#2028)

Co-authored-by: Son NK <son@simplelogin.io>

* Add metrics to rate limit (#2029)

* Order domains alphabetically when retrieving them (#2030)

* Removed unused import

* Remove debug info

---------

Co-authored-by: D-Bao <49440133+D-Bao@users.noreply.github.com>
Co-authored-by: Son Nguyen Kim <son.nguyen@proton.ch>
Co-authored-by: Son NK <son@simplelogin.io>
2024-02-15 15:48:02 +01:00
Adrià Casajús
b462c256d3
Order domains alphabetically when retrieving them (#2030) 2024-02-08 15:36:06 +01:00
Adrià Casajús
f756b04ead
Add metrics to rate limit (#2029) 2024-02-06 11:55:45 +01:00
Son Nguyen Kim
05d18c23cc
support product IDs for the new Mac app (#2028)
Co-authored-by: Son NK <son@simplelogin.io>
2024-02-06 11:54:02 +01:00
Adrià Casajús
4a7c0293f8
Fix: if redis is not configured do not enable rate limit (#2027) 2024-02-05 14:53:01 +01:00
Adrià Casajús
30aaf118e7
Add option for admin to stop trial (#2026) 2024-02-05 13:47:39 +01:00
Adrià Casajús
b6f1cecee9
Move rate limit config to configuration (#2023) 2024-02-01 14:47:15 +01:00
Adrià Casajús
d12e776949
Rate limit alias creation to prevent abuse (#2021)
* Rate limit alias creation to prevent abuse

* Limit in secs

* Calculate bucket time

* fix exception

* Tune limits
2024-01-30 18:29:59 +01:00
Adrià Casajús
761420ece9
Prevent mailboxes that have been disabled from being used again (#2016)
* Prevent mailboxes that have been disabled from being used again

* Improve test

* Get one user since it will be unique
2024-01-23 14:57:40 +01:00
Adrià Casajús
c3848862c3
Fix: limit the id sizes we generate and remove spaces after unidecode 2024-01-22 17:42:58 +01:00
Adrià Casajús
da09db3864
Do not allow free users to create reverse alias to reduce abuse (#2013)
* Do not allow free users to create reverse alias to reduce abuse

* Update format

* Move function under user

* Update tests
2024-01-16 14:51:01 +01:00
Adrià Casajús
44138e25a5
Fix: Dedup the list of mailboxes for an alias (#2010) 2024-01-16 14:50:39 +01:00
Son Nguyen Kim
4a046c5f6f
fix error when user logs out, go back to /dashboard and has the server error (#2003)
* fix error when user logs out, go back to /dashboard and has the server error

* reformat files. Not run ruff on migrations/ and .venv

---------

Co-authored-by: Son NK <son@simplelogin.io>
2024-01-05 14:30:07 +01:00
Son Nguyen Kim
7705fa1c9b
reduce rate limit on /v2/aliases endpoint (#1979)
Co-authored-by: Son NK <son@simplelogin.io>
2023-12-27 16:42:58 +01:00
Adrià Casajús
1dfb0e3356
Require CSRF check on custom alias creation (#1977) 2023-12-20 16:15:01 +01:00
Adrià Casajús
2a9c1c5658
Increase limit for the dashboard and do it by user 2023-12-19 17:27:55 +01:00
Carlos Quintana
dc39ab2de7
chore: remove verbose log (#1971) 2023-12-15 10:39:02 +01:00
Adrià Casajús
fe1c66268b
Allow to use another S3 provider (#1970) 2023-12-14 15:55:37 +01:00
Adrià Casajús
f81f8ca032
Further limit the index endpoint (#1950) 2023-11-21 17:44:33 +01:00
Adrià Casajús
31896ff262
Replace black and flake8 with ruff (#1943) 2023-11-21 16:42:18 +01:00
Adrià Casajús
45575261dc
Rate limit index endpoint (#1948) 2023-11-21 14:42:24 +01:00
Adrià Casajús
627ad302d2
Creating account via partner also canonicalizes email (#1939) 2023-11-08 09:58:01 +01:00
Son Nguyen Kim
75dd3cf925
admin can clone newsletter (#1938)
* admin can clone newsletter

- remove unique constraint on newsletter subject
- admin can clone newsletter

* update coupon image

---------

Co-authored-by: Son NK <son@simplelogin.io>
2023-11-07 14:16:03 +01:00
Adrià Casajús
b68f074783
Add index on message_id for foreign key (#1906)
* Add index on message_id for foreign key

* Revert cron changes
2023-10-05 10:55:29 +02:00
Adrià Casajús
c19b62b878
Add index on created_at for EmailLog (#1898)
(cherry picked from commit ea46ca0af5f6912d17cf7c656f00257cdee191d1)
2023-09-28 18:26:40 +02:00
Adrià Casajús
989a577db6
Allow to get premium partner domains without premium sl domains (#1880)
* Allow to get premium partner domains without premium sl domains

* Set condition on domains
2023-09-13 18:12:47 +02:00
Adrià Casajús
373c30e53b
Schedule deletion of users (#1872)
* Accounts to be scheduled to be deleted cannot receive emails or login

* Create model and create migration for user

* Add test for the cron function

* Move logic to one place

* Use the class name to call the static delete method
2023-09-10 22:11:50 +02:00
Son Nguyen Kim
ff3dbdaad2
add proton.ch to the is_proton check (#1863)
Co-authored-by: Son NK <son@simplelogin.io>
2023-09-04 21:21:39 +02:00
Adrià Casajús
7ec7e06c2b
Move alias transfer util outside the views to make it importable (#1855) 2023-08-31 13:42:44 +02:00
Adrià Casajús
ef90423a35
Fix: Use proper error when linking external partner accounts 2023-08-30 13:49:47 +02:00
Adrià Casajús
c04f5102d6
Fix: Handle email headers as strings if the are Header type (#1850) 2023-08-29 12:37:26 +02:00
Son Nguyen Kim
5714403976
Can use generic subject without pgp (#1847)
* improve wording for hide my subject option

* can use generic subject on a non-pgp mailbox

---------

Co-authored-by: Son NK <son@simplelogin.io>
2023-08-24 22:47:31 +02:00
Carlos Quintana
40ff4604c8
fix: handle Proton account not validated case (#1842) 2023-08-18 15:59:46 +02:00
Son Nguyen Kim
0435c745fd
disable the PGP section if the mailbox is proton and not has PGP enabled (#1841)
* disable the PGP section if the mailbox is proton and not has PGP enabled

* fix format

---------

Co-authored-by: Son NK <son@simplelogin.io>
2023-08-09 09:56:53 +02:00
Adrià Casajús
4bf925fe6f
Revert contact creation (#1836)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-08-04 14:01:21 +02:00
Carlos Quintana
0e82801512
chore: add upcloud monitoring (#1835)
* chore: add upcloud monitoring

* Added db_role to new_relic metrics

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-08-04 12:19:00 +02:00
Adrià Casajús
9ab3695d36
Fix: Do not lowercase by default contact emails (#1834)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-08-04 10:36:13 +02:00
Adrià Casajús
6e4f6fe540
Sanitize alias, contacts, mailboxes and users before creating them (#1829)
* Sanitize alias, contacts, mailboxes and users before creating them

* Updated comments and moved crons to run when load is low

* Run the stats at the same time as previously

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-08-03 10:20:25 +02:00
Adrià Casajús
e9e863807c
Add missing indexes (#1824)
* Rate limit the sudo route

* Add missing indexes

* Updated index

* Update index creation to run with concurrent

* With autocommit block

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-29 10:03:31 +02:00
Adrià Casajús
c4003b07ac
Rate limit the sudo route (#1823)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-26 12:56:06 +02:00
Adrià Casajús
d8943cf126
Fix: Allow to create more than one api key if the user has more than one (#1822)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-25 17:15:18 +02:00
Adrià Casajús
04d92b7f23
Fix: Use MIMEText for text contents (#1801)
* Fix: For badly formatted messages use MIMEText

* Fix: For badly formatted messages use MIMEText

* fix test

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-11 16:48:01 +02:00
Adrià Casajús
cb900ed057
Fix: For badly formatted messages use MIMEText (#1800)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-11 16:23:37 +02:00
Adrià Casajús
516072fd99
Fix: save retries to disk (#1799)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-11 10:59:24 +02:00
Adrià Casajús
e2dbf8d48d
Avoid sending long encoded subject to sentry (#1798)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-07-10 14:41:42 +02:00
Adrià Casajús
d62bff8e46
Add rate limit and maximum amount of api keys (#1788)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-29 17:21:00 +02:00
Adrià Casajús
ac9d550069
Fix: delete_header has no return value (#1786)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-27 14:42:52 +02:00
Adrià Casajús
daec781ffc
Fix unsubscribe header manipulation (#1785)
* Added debug statements to find out unsubscribe issues

* Add List-Unsubscribe headers to preserve list

* Cleanup debug messages

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-27 11:18:44 +02:00
Son Nguyen Kim
501c625ddf
set default alias suffix to word (#1765)
Co-authored-by: Son NK <son@Sons-MacBook-Air-2.local>
2023-06-27 11:07:02 +02:00
Adrià Casajús
d3aae31d45
Preserve original from header in X-SimpleLogin-Original-From (#1784)
* Preserve original from in the headers

* Update the settings page

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-23 12:43:06 +02:00
Adrià Casajús
76b05e0d64
Preserve original sender and authentication results if the original email is preserved in the alias (#1780)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-22 10:40:32 +02:00
Adrià Casajús
f046b2270c
Fix: send also mailbox email to verify so that mailbox changes are not allowed (#1777)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-21 18:56:22 +02:00
Adrià Casajús
03c67ead44
Do not show the default domain twice (#1772)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-21 18:56:03 +02:00
Adrià Casajús
37ffe4d5fe
Fix: Always include default domain in the list of domains (#1768)
* Fix: Always include default domain in the list of domains

* Add premium test

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-06 15:55:10 +02:00
Adrià Casajús
689ef3a579
Check if the domain has a deleted alias (#1764)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-01 17:33:58 +02:00
Adrià Casajús
495d544505
Only retry n times each message (#1759)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-01 10:59:02 +02:00
Adrià Casajús
a539428607
Fix: If default domain is premium for free users do not offer it as an option (#1763)
* Fix: If default domain is premium for free users do not offer it as an option

* Refactored into simpler logic

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-06-01 10:46:25 +02:00
Adrià Casajús
9d9e5fcab6
Fix: If the default domain is hidden do not return it (#1761)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-31 17:59:49 +02:00
Adrià Casajús
ff33392398
Fix: use incorrect model to access profile picture path (#1760)
(cherry picked from commit e875f1dd40fe726f6e83aaa833f65eb9e10f7e94)

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-31 17:21:30 +02:00
Adrià Casajús
85964f283e
Add timeout to any outbound connection (#1756)
* Add timeout to any outbound connection

* Change log message to error

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-31 14:19:43 +02:00
Carlos Quintana
d30183bbda
fix: remove user password from export user data (#1758) 2023-05-31 09:40:20 +02:00
Adrià Casajús
ed66c7306b
Fix typo (#1755)
Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-29 17:50:41 +02:00
Adrià Casajús
07bb658310
Show the default domain for creating aliases even if it's not requested by a partner (#1754)
* Show the default domain in the suffixes even if it's not allowed

* Simplify logic

* Reformat

* Simplified logic

* Remove unused function

* Added test to validate suffixes

* Ensure we catch prefixes in test

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-29 16:40:04 +02:00
Adrià Casajús
e43a2dd34d
Have subscription callback whenever a subscription changes (#1748)
* Have subscription callback whenever a subscription changes

* Fixed tests

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-26 15:33:55 +02:00
Adrià Casajús
3de83f2f05
Add toggle to check if a user is premium without the partner subscription (#1739)
* Add toggle to check if a user is premium without the partner subscription

* fix test

* Parter created users do not have a newsletter alias id

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-15 12:34:58 +02:00
Adrià Casajús
e4d4317988
Various fixes (#1733)
* Reset all password tokens on password reset

* Added csrf validation on email change request and validation

* Return the same wether is a valid email or not

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-10 15:31:30 +02:00
Adrià Casajús
6dfb6bb3e4
Revert "Add code verification for creating mailboxes (#1725)" (#1727)
This reverts commit a5e7da10dd.

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-09 18:04:04 +02:00
Adrià Casajús
a5e7da10dd
Add code verification for creating mailboxes (#1725)
* Add code verification for creating mailboxes

* Added validation checks

* Use exceptions

* Added delete to the mailbox utils

* Fix test

* Update package.lock

* Fix delete error

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-09 16:35:58 +02:00
Adrià Casajús
5ddbca05b2
Check users aren't using an alias as their link email address for partner links (#1724)
(cherry picked from commit 93e24cb4239b812d46f119a982edd12de2406802)

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-08 18:47:10 +02:00
Adrià Casajús
7cb7b48845
Ensure coupons are only used once (#1718)
* Ensure coupons are only used once

* Update test to handle redirect

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-05-03 16:18:46 +02:00
Son Nguyen Kim
6276ad4419
Stats endpoint (#1716)
* update some dependencies: newrelic, gevent

that isn't compatible with python 3.11 on mac

* update package-lock using npm 9.6.4 and node 20.0

* Add GET /api/stats

* update pytest

---------

Co-authored-by: Son Nguyen Kim <son@Sons-MacBook-Air-2.local>
2023-05-03 10:15:47 +02:00
Adrià Casajús
bec8cb2292
Alias domain as contact domain (#1689)
* Use the alias domain for contacts

* Check there are not duplicate emails

* Check also in trash

* Use helper

* Set VERP for the forward phase to the contact domain

* Add pgp_fingerprint as index for contacts

* Removed check trash

* Only use reply domains for sl domains

* Configure via db wether the domain can be used as a reverse_domain

* Fix: typo

* reverse logic

* fix migration

* fix test

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
Co-authored-by: Son <nguyenkims@users.noreply.github.com>
2023-04-20 12:14:53 +02:00
Adrià Casajús
7f23533c64
Fix sever typo (#1701)
* Fix: typo

* Limit the name to 100 chars

* Fix migration

---------

Co-authored-by: Adrià Casajús <adria.casajus@proton.ch>
2023-04-20 11:06:59 +02:00