app-MAIL-temp/README.md

565 lines
19 KiB
Markdown
Raw Normal View History

2020-01-01 10:43:44 +01:00
SimpleLogin | Privacy-First Email Forwarding/Alias and Identity Provider service
2019-12-18 17:10:10 +01:00
---
https://simplelogin.io
2020-01-01 10:38:19 +01:00
# Quick start
If you have Docker installed, run the following command to start SimpleLogin local server:
```bash
docker run -it --rm \
-e RESET_DB=true \
-e CONFIG=/code/.env.example \
-p 7777:7777 \
simplelogin/app python server.py
```
Then open http://localhost:7777, you should be able to login with `john@wick.com/password` account!
To use SimpleLogin email aliases, you need to deploy it on your server with some DNS setup though,
2020-01-01 10:43:44 +01:00
the following section will show a step-by-step guide on how to get your own email forwarder service!
# Introduction
> Yet another email forwarding service?
In some way yes... However, SimpleLogin is a bit different because:
- it's fully open source: both the server and client code (browser extension, JS library) are open source so anyone can freely inspect and (hopefully) improve the code.
- not just email alias: SimpleLogin is a privacy-first and developer-friendly identity provider that:
- offers privacy for users
- is simple to use for developers. SimpleLogin is a privacy-focused alternative to the "Login with Facebook/Google/Twitter" buttons.
- the only email alias solution that is `self-hostable`: with our detailed self-hosting instructions and most of components running as Docker container, anyone who knows how to `ssh` is able to deploy SimpleLogin on their server.
- plenty of features: custom domain, browser extension, alias activity, OAuth libraries, etc.
- written in Python 🐍 😅 this is not a difference per se but hey I never found a Python email server so feel free to tweak this one if you want to use Python for handling emails.
2020-01-01 10:38:19 +01:00
2019-12-19 21:19:16 +01:00
# Table of Contents
[1. General Architecture](#general-architecture)
[2. Self Hosting](#self-hosting)
[3. Contributing Guide](#contributing)
2019-12-18 17:10:10 +01:00
## General Architecture
![](docs/archi.png)
SimpleLogin backend consists of 2 main components:
- the `webapp` used by several clients: web UI (the dashboard), browser extension (Chrome & Firefox for now), OAuth clients (apps that integrate "Login with SimpleLogin" button) and mobile app (work in progress).
- the `email handler`: implements the email forwarding (i.e. alias receiving email) and email sending (i.e. alias sending email).
2019-12-18 17:10:10 +01:00
## Self hosting
### Prerequisites
2019-12-21 12:58:33 +01:00
- a Linux server (either a VM or dedicated server). This doc shows the setup for Ubuntu 18.04 LTS but the steps could be adapted for other popular Linux distributions. As most of components run as Docker container and Docker can be a bit heavy, having at least 2 GB of RAM is recommended. The server needs to have the port 25 (email), 80, 443 (for the webapp), 22 (so you can ssh into it) open.
2019-12-18 17:10:10 +01:00
- a domain that you can config the DNS. It could be a sub-domain. In the rest of the doc, let's say it's `mydomain.com` for the email and `app.mydomain.com` for SimpleLogin webapp. Please make sure to replace these values by your domain name whenever they appear in the doc.
2019-12-18 17:10:10 +01:00
2019-12-21 12:58:33 +01:00
- [Optional]: a dedicated Postgres database. If you don't want to manage and maintain a Postgres database, you can use managed services proposed by some cloud providers. Otherwise this guide will show how to run a Postgres database using Docker. Database is not well-known to be run inside Docker but this is probably fine if you don't have thousands of email addresses.
2019-12-18 17:10:10 +01:00
2019-12-21 12:58:33 +01:00
- [Optional] AWS S3, Sentry, Google/Facebook/Github developer accounts. These are necessary only if you want to activate these options.
2019-12-18 17:10:10 +01:00
2019-12-21 12:58:33 +01:00
All the below steps, except for the DNS ones that are usually done inside your domain registrar interface, are done on your server. The commands are to run with `bash` (or any bash-compatible shell like `zsh`) being the shell. If you use other shells like `fish`, please make sure to adapt the commands.
2019-12-18 17:10:10 +01:00
### DKIM
From Wikipedia https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail
> DomainKeys Identified Mail (DKIM) is an email authentication method designed to detect forged sender addresses in emails (email spoofing), a technique often used in phishing and email spam.
2019-12-21 12:58:33 +01:00
Setting up DKIM is highly recommended to reduce the chance your emails ending up in the recipient's Spam folder.
2019-12-18 17:10:10 +01:00
First you need to generate a private and public key for DKIM:
```bash
openssl genrsa -out dkim.key 1024
openssl rsa -in dkim.key -pubout -out dkim.pub.key
```
You will need the files `dkim.key` and `dkim.pub.key` for the next steps.
2019-12-21 12:58:33 +01:00
For email gurus, we have chosen 1024 key length instead of 2048 for DNS simplicity as some registrars don't play well with long TXT record.
2019-12-18 17:10:10 +01:00
### DNS
Please note that DNS changes could take up to 24 hours to propagate. In practice, it's a lot faster though (~1 minute or so in our test).
#### MX record
Create a **MX record** that points `mydomain.com` to `app.mydomain.com` with priority 10.
To verify if the DNS works, `dig mydomain.com mx` should contain the following in the result.
```
mydomain.com. 3600 IN MX 10 app.mydomain.com.
```
#### A record
An **A record** that points `app.mydomain.com` to your server IP. To verify, `dig app.mydomain.com a` should return your server IP.
#### DKIM
Set up DKIM by adding a TXT record for `dkim._domainkey.mydomain.com` with the following value:
```
2019-12-21 12:58:33 +01:00
v=DKIM1; k=rsa; p=PUBLIC_KEY
2019-12-18 17:10:10 +01:00
```
2019-12-21 12:58:33 +01:00
with `PUBLIC_KEY` being your `dkim.pub.key` but
2019-12-18 17:10:10 +01:00
- remove the `-----BEGIN PUBLIC KEY-----` and `-----END PUBLIC KEY-----`
- join all the lines on a single line.
2019-12-21 12:58:33 +01:00
For example, if your `dkim.pub.key` is
2019-12-18 17:10:10 +01:00
```
-----BEGIN PUBLIC KEY-----
ab
cd
ef
gh
-----END PUBLIC KEY-----
```
2019-12-21 12:58:33 +01:00
then the `PUBLIC_KEY` would be `abcdefgh`.
2019-12-18 17:10:10 +01:00
To verify, `dig dkim._domainkey.mydomain.com txt` should return the above value.
#### SPF
From Wikipedia https://en.wikipedia.org/wiki/Sender_Policy_Framework
> Sender Policy Framework (SPF) is an email authentication method designed to detect forging sender addresses during the delivery of the email
Similar to DKIM, setting up SPF is highly recommended.
2019-12-18 17:30:31 +01:00
Add a TXT record for `mydomain.com` with the value `v=spf1 mx -all`. What it means is only your server can send email with `@mydomain.com` domain. To verify, you can use `dig mydomain.com txt`
2019-12-18 17:10:10 +01:00
#### DMARC (optional) TODO
### Docker
Now the boring DNS stuffs are done, let's do something more fun!
Please follow the steps on [Docker CE for Ubuntu](https://docs.docker.com/v17.12/install/linux/docker-ce/ubuntu/) to install Docker on the server.
Tips: if you want to run Docker without the `sudo` prefix, add your account to `docker` group:
```bash
sudo usermod -a -G docker $USER
```
### Prepare the Docker network
This Docker network will be used by the other Docker containers run in the next steps.
Later, we will setup Postfix to authorize this network.
```bash
docker network create -d bridge \
--subnet=1.1.1.0/24 \
--gateway=1.1.1.1 \
sl-network
```
### Postgres
This section shows how to run a Postgres database using Docker. At the end of this section, you will have a database username and password which are being referred to the next steps.
2019-12-18 17:10:10 +01:00
If you have already had a Postgres database in use, you can skip this section and just copy the database configuration (i.e. host, port, username, password, database name).
2019-12-18 17:10:10 +01:00
Run a Postgres Docker container as your Postgres database server. Make sure to replace `myuser` and `mypassword` with something more secret 😎.
2019-12-18 17:10:10 +01:00
```bash
docker run -d \
--name sl-db \
-e POSTGRES_PASSWORD=mypassword \
-e POSTGRES_USER=myuser \
-e POSTGRES_DB=simplelogin \
--network="sl-network" \
postgres
```
To test whether the database operates correctly or not, run the following command:
```bash
docker exec -it sl-db psql -U myuser simplelogin
```
you should be logged in the postgres console.
2019-12-18 17:10:10 +01:00
### Postfix
Install `postfix` and `postfix-pgsql`. The latter is used to connect Postfix and the Postgres database in the next steps.
2019-12-18 17:10:10 +01:00
```bash
sudo apt-get install -y postfix postfix-pgsql
```
Choose "Internet Site" in Postfix installation window then keep using the proposed value as *System mail name* in the next window.
Run the following commands to setup Postfix. Make sure to replace `mydomain.com` with the appropriate value of your domain.
2019-12-18 17:10:10 +01:00
```bash
sudo postconf -e 'myhostname = app.mydomain.com'
sudo postconf -e 'mydomain = mydomain.com'
sudo postconf -e 'myorigin = mydomain.com'
sudo postconf -e 'mydestination = localhost'
sudo postconf -e 'mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 1.1.1.0/24'
sudo postconf -e 'relay_domains = pgsql:/etc/postfix/pgsql-relay-domains.cf'
sudo postconf -e 'transport_maps = pgsql:/etc/postfix/pgsql-transport-maps.cf'
```
Create the `relay-domains` file at `/etc/postfix/pgsql-relay-domains.cf` and put the following content. Make sure that the database config is correctly set and replace `mydomain.com` with your domain.
2019-12-18 17:10:10 +01:00
```
# postgres config
hosts = localhost
user = myuser
password = mypassword
dbname = simplelogin
query = SELECT domain FROM custom_domain WHERE domain='%s' AND verified=true UNION SELECT '%s' WHERE '%s' = 'mydomain.com' LIMIT 1;
```
Create the `transport-maps` file at `/etc/postfix/pgsql-transport-maps.cf` and put the following lines. Make sure that the database config is correctly set and replace `mydomain.com` with your domain.
2019-12-18 17:10:10 +01:00
```
# postgres config
hosts = localhost
user = myuser
password = mypassword
dbname = simplelogin
# forward to smtp:127.0.0.1:20381 for custom domain AND email domain
query = SELECT 'smtp:127.0.0.1:20381' FROM custom_domain WHERE domain = '%s' AND verified=true UNION SELECT 'smtp:127.0.0.1:20381' WHERE '%s' = 'mydomain.com' LIMIT 1;
```
Finally, restart Postfix
2019-12-18 17:10:10 +01:00
> sudo systemctl restart postfix
### Run SimpleLogin Docker containers
2019-12-21 12:58:33 +01:00
To run the server, you need a config file. Please have a look at [config example](./.env.example) for an example to create one. Some parameters are optional and are commented out by default. Some have "dummy" values, fill them up if you want to enable these features (Paddle, AWS).
2019-12-18 17:10:10 +01:00
Let's put your config file at `~/simplelogin.env`.
2019-12-27 11:49:33 +01:00
Make sure to update the following variables and replace these values by yours.
2019-12-18 17:10:10 +01:00
```.env
# Server url
URL=http://app.mydomain.com
EMAIL_DOMAIN=mydomain.com
SUPPORT_EMAIL=support@mydomain.com
EMAIL_SERVERS_WITH_PRIORITY=[(10, "app.mydomain.com.")]
DKIM_PRIVATE_KEY_PATH=/dkim.key
2019-12-27 23:35:28 +01:00
DKIM_PUBLIC_KEY_PATH=/dkim.pub.key
2019-12-27 11:49:33 +01:00
DB_URI=postgresql://myuser:mypassword@sl-db:5432/simplelogin
2019-12-18 17:10:10 +01:00
# optional, to have more choices for random alias.
WORDS_FILE_PATH=local_data/words_alpha.txt
```
Before running the webapp, you need to prepare the database by running the migration
```bash
2019-12-27 11:49:33 +01:00
docker run --rm \
2019-12-18 17:10:10 +01:00
--name sl-migration \
-v $(pwd)/dkim.key:/dkim.key \
2019-12-27 23:35:28 +01:00
-v $(pwd)/dkim.pub.key:/dkim.pub.key \
-v $(pwd)/simplelogin.env:/code/.env \
2019-12-18 17:10:10 +01:00
--network="sl-network" \
simplelogin/app flask db upgrade
```
This command could take a while to download the `simplelogin/app` docker image.
Now, it's time to run the `webapp` container!
2019-12-18 17:10:10 +01:00
```bash
docker run -d \
--name sl-app \
-v $(pwd)/simplelogin.env:/code/.env \
2019-12-18 17:10:10 +01:00
-v $(pwd)/dkim.key:/dkim.key \
2019-12-27 23:35:28 +01:00
-v $(pwd)/dkim.pub.key:/dkim.pub.key \
2019-12-18 17:10:10 +01:00
-p 7777:7777 \
--network="sl-network" \
simplelogin/app
```
Next run the `email handler`
2019-12-18 17:10:10 +01:00
```bash
docker run -d \
--name sl-email \
-v $(pwd)/simplelogin.env:/code/.env \
2019-12-18 17:10:10 +01:00
-v $(pwd)/dkim.key:/dkim.key \
2019-12-27 23:35:28 +01:00
-v $(pwd)/dkim.pub.key:/dkim.pub.key \
2019-12-18 17:10:10 +01:00
-p 20381:20381 \
--network="sl-network" \
simplelogin/app python email_handler.py
```
[Optional] If you want to run the cronjob:
```bash
docker run -d \
--name sl-cron \
-v $(pwd)/simplelogin.env:/code/.env \
2019-12-18 17:10:10 +01:00
-v $(pwd)/dkim.key:/dkim.key \
2019-12-27 23:35:28 +01:00
-v $(pwd)/dkim.pub.key:/dkim.pub.key \
2019-12-18 17:10:10 +01:00
--network="sl-network" \
simplelogin/app yacron -c /code/crontab.yml
```
### Nginx
Install Nginx
```bash
sudo apt-get install -y nginx
```
Then, create `/etc/nginx/sites-enabled/simplelogin` with the following lines:
2019-12-18 17:10:10 +01:00
```
server {
server_name app.mydomain.com;
location / {
proxy_pass http://localhost:7777;
}
}
```
Reload Nginx with the command below
2019-12-18 17:10:10 +01:00
```bash
sudo systemctl reload nginx
```
At this step, you should also setup the SSL for Nginx. [Certbot](https://certbot.eff.org/lets-encrypt/ubuntuxenial-nginx) can be a good option if you want a free SSL certificate.
### Enjoy!
If all of the above steps are successful, open http://app.mydomain.com/ and create your first account!
2019-12-18 17:10:10 +01:00
## Contributing
2019-12-19 21:19:16 +01:00
All work on SimpleLogin happens directly on GitHub.
### Run code locally
The project uses Python 3.7+. First, install all dependencies by running the following command. Feel free to use `virtualenv` or similar tools to isolate development environment.
2019-12-19 21:19:16 +01:00
```bash
pip3 install -r requirements.txt
```
Then make sure all tests pass
```bash
pytest
```
2019-12-18 17:10:10 +01:00
To run the code locally, please create a local setting file based on `.env.example`:
create BaseForm to enable CSRF register page redirect user to dashboard if they are logged in enable csrf for login page Set models more strict bootstrap developer page add helper method to ModelMixin, remove CRUDMixin display list of clients on developer index, add copy client-secret to clipboard using clipboardjs add toastr and use jquery non slim display a toast when user copies the client-secret create new client, generate client-id using unidecode client detail page: can edit client add delete client implement /oauth/authorize and /oauth/allow-deny implement /oauth/token add /oauth/user_info endpoint handle scopes: wip take into account scope: display scope, return user data according to scope create virtual-domain, gen email, client_user model WIP create authorize_nonlogin_user page user can choose to generate a new email no need to interfere with root logger log for before and after request if user has already allowed a client: generate a auth-code and redirect user to client get_user_info takes into account gen email display list of clients that have user has authorised use yk-client domain instead of localhost as cookie depends on the domain name use wtforms instead of flask_wtf Dockerfile delete virtual domain EMAIL_DOMAIN can come from env var bind to host 0.0.0.0 fix signup error: use session as default csrf_context rename yourkey to simplelogin add python-dotenv, ipython, sqlalchemy_utils create DB_URI, FLASK_SECRET. Load config from CONFIG file if exist add shortcuts to logging create shell add psycopg2 do not add local data in Dockerfile add drop_db into shell add shell.prepare_db() fix prepare_db setup sentry copy assets from tabler/dist add icon downloaded from https://commons.wikimedia.org/wiki/File:Simpleicons_Interface_key-tool-1.svg integrate tabler - login and register page add favicon template: default, header. Use gravatar for user avatar url use default template for dashboard, developer page use another icon add clipboard and notie prettify dashboard add notie css add fake gen email and client-user prettify list client page, use notie for toast add email, name scope to new client display client scope in client list prettify new-client, client-detail add sentry-sdk and blinker add arrow, add dt jinja filter, prettify logout, dashboard comment "last used" in dashboard for now prettify date display add copy email to clipboard to dashboard use "users" as table name for User as "user" is reserved key in postgres call prepare_db() when creating new db error page 400, 401, 403, 404 prettify authorize_login_user create already_authorize.html for user who has already authorized a client user can generate new email display all other generated emails add ENV variable, only reset DB when ENV=local fix: not return other users gen emails display nb users for each client refactor shell: remove prepare_db() add sendgrid add /favicon.ico route add new config: URL, SUPPORT_EMAIL, SENDGRID_API_KEY user needs to activate their account before login create copy button on dashboard client can have multiple redirect uris, in client detail can add/remove redirect-uri, use redirect_uri passed in /authorize refactor: move get_user_info into ClientUser model dashboard: display all apps, all generated emails add "id" into user_info add trigger email button invalidate the session at each new version by changing the secret centralize Client creation into Client.create_new user can enable/disable email forwarding setup auto dismiss alert: just add .alert-auto-dismiss move name down in register form add shell.add_real_data move blueprint template to its own package prettify authorize page for non-authenticated user update readme, return error if not redirect_uri add flask-wtf, use psycopg2-binary use flask-wtf FlaskForm instead of Form rename email -> email_utils add AWS_REGION, BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY to config add s3 module add File model, add Client.icon_id handle client icon update can create client with icon display client icon in client list page add Client.home_url take into account Client.home_url add boto3 register: ask name first only show "trigger test email" if email forwarding is enabled display gen email in alphabetical order, client in client.name alphabetical order better error page the modal does not get close when user clicks outside of modal add Client.published column discover page that displays all published Client add missing bootstrap.bundle.min.js.map developer can publish/unpublish their app in discover use notie for display flash message create hotmail account fix missing jquery add footer, add global jinja2 variable strengthen model: use nullable=False whenever possible, rename client_id to oauth_client_id, client_secret to oauth_client_secret add flask-migrate init migrate 1st migrate version fix rename client_id -> oauth_client_id prettify UI use flask_migrate.upgrade() instead of db.create_all() make sure requirejs.config is called for all page enable sentry for js, use uppercase for global jinja2 variables add flask-admin add User.is_admin column setup flask admin, only accessible to admin user fix migration: add server_default replace session[redirect_after_login] by "next" request args add pyproject.toml: ignore migrations/ in black add register waiting_activation_email page better email wording add pytest add get_host_name_and_scheme and tests example fail test fix test fix client-id display add flask-cors /user_info supports cors, add /me as /user_info synonym return client in /me support implicit flow no need to use with "app.app_context()" add watchtower to requirement add param ENABLE_CLOUDWATCH, CLOUDWATCH_LOG_GROUP, CLOUDWATCH_LOG_STREAM add cloudwatch logger if cloudwatch is enabled add 500 error page add help text for list of used client display list of app/website that an email has been used click on client name brings to client detail page create style.css to add additional style, append its url with the current sha1 to avoid cache POC on how to send email using postfix add sqlalchemy-utils use arrow instead of datetime add new params STRIPE_API, STRIPE_YEARLY_SKU, STRIPE_MONTHLY_PLAN show full error in local add plan, plan_expiration to User, need to create enum directly in migration script, cf https://github.com/sqlalchemy/alembic/issues/67 reformat all html files: use space instead of tab new user will have trial plan for 15 days add new param MAX_NB_EMAIL_FREE_PLAN only user with enough quota can create new email if user cannot create new gen email, pick randomly one from existing gen emails. Use flush instead of commit rename STRIPE_YEARLY_SKU -> STRIPE_YEARLY_PLAN open client page in discover in a new tab add stripe not logging /static call: disable flask logging, replace by after_request add param STRIPE_SECRET_KEY add 3 columns stripe_customer_id, stripe_card_token, stripe_subscription_id user can upgrade their pricing add setting page as coming-soon add GenEmail, ClientUser to admin ignore /admin/static logging add more fake data add ondelete="cascade" whenever possible rename plan_expiration -> trial_expiration reset migration: delete old migrations, create new one rename test_send_email -> poc_send_email to avoid the file being called by pytest add new param LYRA_ANALYTICS_ID, add lyra analytics add how to create new migration into readme add drift to base.html notify admin when new user signs up or pays subscription log exception in case of 500 use sendgrid to notify admin add alias /userinfo to user_info endpoint add change_password to shell add info on how payment is handled invite user to retry if card not working remove drift and add "contact us" link move poc_send_email into poc/ support getting client-id, client-secret from form-data in addition to basic auth client-id, client-secret is passed in form-data by passport-oauth2 for ex add jwtRS256 private and public key add jwk-jws-jwt poc add new param OPENID_PRIVATE_KEY_PATH, OPENID_PRIVATE_KEY_PATH add scope, redirect_url to AuthorizationCode and OauthToken take into scope when creating oauth-token, authorization-code add jwcrypto add jose_utils: make_id_token and verify_id_token add &scope to redirect uri add "email_verified": True into user_info fix user not activated add /oauth2 as alias for /oauth handle case where scope and state are empty remove threaded=False Use Email Alias as wording remove help text user can re-send activation email add "expired" into ActivationCode Handle the case activation code is expired reformat: use form.validate_on_submit instead of request.method == post && form.validate use error text instead of flash() display client oauth-id and oauth-secret on client detail page not display oauth-secret on client listing fix expiration check improve page title, footer add /jwks and /.well-known/openid-configuration init properly tests, fix blueprint conflict bug in flask-admin create oauth_models module rename Scope -> ScopeE to distinguish with Scope DB model set app.url_map.strict_slashes = False use ScopeE instead of SCOPE_NAME, ... support access_token passed as args in /userinfo merge /allow-deny into /authorize improve wording take into account the case response_type=code and openid is in scope take into account response_type=id_token, id_token token, id_token code make sure to use in-memory db in test fix scope can be null allow cross_origin for /.well-known/openid-configuration and /jwks fix footer link center authorize form rename trial_expiration to plan_expiration move stripe init to create_app() use real email to be able to receive email notification add user.profile_picture_id column use user profile picture and fallback to gravatar use nguyenkims+local@gm to distinguish with staging handle plan cancel, reactivation, user profile update fix can_create_new_email create cron.py that set plan to free when expired add crontab.yml add yacron use notify_admin instead of LOG.error add ResetPasswordCode model user can change password in setting increase display time for notie add forgot_password page If login error: redirect to this page upon success login. hide discover tab add column user.is_developer only show developer menu to developer comment out the publish button set local user to developer make sure only developer can access /developer blueprint User is invited to upgrade if they are in free plan or their trial ends soon not sending email when in local mode create Partner model create become partner page use normal error handling on local fix migration add "import sqlalchemy_utils" into migration template small refactoring on setting page handle promo code. TODO: add migration file add migration for user.promo_codes move email alias on top of apps in dashboard add introjs move encode_url to utils create GenEmail.create_new_gen_email create a first alias mail to show user how to use when they login show intro when user visits the website the first time fix register
2019-07-02 09:20:12 +02:00
2019-12-09 22:26:58 +01:00
```
cp .env.example .env
create BaseForm to enable CSRF register page redirect user to dashboard if they are logged in enable csrf for login page Set models more strict bootstrap developer page add helper method to ModelMixin, remove CRUDMixin display list of clients on developer index, add copy client-secret to clipboard using clipboardjs add toastr and use jquery non slim display a toast when user copies the client-secret create new client, generate client-id using unidecode client detail page: can edit client add delete client implement /oauth/authorize and /oauth/allow-deny implement /oauth/token add /oauth/user_info endpoint handle scopes: wip take into account scope: display scope, return user data according to scope create virtual-domain, gen email, client_user model WIP create authorize_nonlogin_user page user can choose to generate a new email no need to interfere with root logger log for before and after request if user has already allowed a client: generate a auth-code and redirect user to client get_user_info takes into account gen email display list of clients that have user has authorised use yk-client domain instead of localhost as cookie depends on the domain name use wtforms instead of flask_wtf Dockerfile delete virtual domain EMAIL_DOMAIN can come from env var bind to host 0.0.0.0 fix signup error: use session as default csrf_context rename yourkey to simplelogin add python-dotenv, ipython, sqlalchemy_utils create DB_URI, FLASK_SECRET. Load config from CONFIG file if exist add shortcuts to logging create shell add psycopg2 do not add local data in Dockerfile add drop_db into shell add shell.prepare_db() fix prepare_db setup sentry copy assets from tabler/dist add icon downloaded from https://commons.wikimedia.org/wiki/File:Simpleicons_Interface_key-tool-1.svg integrate tabler - login and register page add favicon template: default, header. Use gravatar for user avatar url use default template for dashboard, developer page use another icon add clipboard and notie prettify dashboard add notie css add fake gen email and client-user prettify list client page, use notie for toast add email, name scope to new client display client scope in client list prettify new-client, client-detail add sentry-sdk and blinker add arrow, add dt jinja filter, prettify logout, dashboard comment "last used" in dashboard for now prettify date display add copy email to clipboard to dashboard use "users" as table name for User as "user" is reserved key in postgres call prepare_db() when creating new db error page 400, 401, 403, 404 prettify authorize_login_user create already_authorize.html for user who has already authorized a client user can generate new email display all other generated emails add ENV variable, only reset DB when ENV=local fix: not return other users gen emails display nb users for each client refactor shell: remove prepare_db() add sendgrid add /favicon.ico route add new config: URL, SUPPORT_EMAIL, SENDGRID_API_KEY user needs to activate their account before login create copy button on dashboard client can have multiple redirect uris, in client detail can add/remove redirect-uri, use redirect_uri passed in /authorize refactor: move get_user_info into ClientUser model dashboard: display all apps, all generated emails add "id" into user_info add trigger email button invalidate the session at each new version by changing the secret centralize Client creation into Client.create_new user can enable/disable email forwarding setup auto dismiss alert: just add .alert-auto-dismiss move name down in register form add shell.add_real_data move blueprint template to its own package prettify authorize page for non-authenticated user update readme, return error if not redirect_uri add flask-wtf, use psycopg2-binary use flask-wtf FlaskForm instead of Form rename email -> email_utils add AWS_REGION, BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY to config add s3 module add File model, add Client.icon_id handle client icon update can create client with icon display client icon in client list page add Client.home_url take into account Client.home_url add boto3 register: ask name first only show "trigger test email" if email forwarding is enabled display gen email in alphabetical order, client in client.name alphabetical order better error page the modal does not get close when user clicks outside of modal add Client.published column discover page that displays all published Client add missing bootstrap.bundle.min.js.map developer can publish/unpublish their app in discover use notie for display flash message create hotmail account fix missing jquery add footer, add global jinja2 variable strengthen model: use nullable=False whenever possible, rename client_id to oauth_client_id, client_secret to oauth_client_secret add flask-migrate init migrate 1st migrate version fix rename client_id -> oauth_client_id prettify UI use flask_migrate.upgrade() instead of db.create_all() make sure requirejs.config is called for all page enable sentry for js, use uppercase for global jinja2 variables add flask-admin add User.is_admin column setup flask admin, only accessible to admin user fix migration: add server_default replace session[redirect_after_login] by "next" request args add pyproject.toml: ignore migrations/ in black add register waiting_activation_email page better email wording add pytest add get_host_name_and_scheme and tests example fail test fix test fix client-id display add flask-cors /user_info supports cors, add /me as /user_info synonym return client in /me support implicit flow no need to use with "app.app_context()" add watchtower to requirement add param ENABLE_CLOUDWATCH, CLOUDWATCH_LOG_GROUP, CLOUDWATCH_LOG_STREAM add cloudwatch logger if cloudwatch is enabled add 500 error page add help text for list of used client display list of app/website that an email has been used click on client name brings to client detail page create style.css to add additional style, append its url with the current sha1 to avoid cache POC on how to send email using postfix add sqlalchemy-utils use arrow instead of datetime add new params STRIPE_API, STRIPE_YEARLY_SKU, STRIPE_MONTHLY_PLAN show full error in local add plan, plan_expiration to User, need to create enum directly in migration script, cf https://github.com/sqlalchemy/alembic/issues/67 reformat all html files: use space instead of tab new user will have trial plan for 15 days add new param MAX_NB_EMAIL_FREE_PLAN only user with enough quota can create new email if user cannot create new gen email, pick randomly one from existing gen emails. Use flush instead of commit rename STRIPE_YEARLY_SKU -> STRIPE_YEARLY_PLAN open client page in discover in a new tab add stripe not logging /static call: disable flask logging, replace by after_request add param STRIPE_SECRET_KEY add 3 columns stripe_customer_id, stripe_card_token, stripe_subscription_id user can upgrade their pricing add setting page as coming-soon add GenEmail, ClientUser to admin ignore /admin/static logging add more fake data add ondelete="cascade" whenever possible rename plan_expiration -> trial_expiration reset migration: delete old migrations, create new one rename test_send_email -> poc_send_email to avoid the file being called by pytest add new param LYRA_ANALYTICS_ID, add lyra analytics add how to create new migration into readme add drift to base.html notify admin when new user signs up or pays subscription log exception in case of 500 use sendgrid to notify admin add alias /userinfo to user_info endpoint add change_password to shell add info on how payment is handled invite user to retry if card not working remove drift and add "contact us" link move poc_send_email into poc/ support getting client-id, client-secret from form-data in addition to basic auth client-id, client-secret is passed in form-data by passport-oauth2 for ex add jwtRS256 private and public key add jwk-jws-jwt poc add new param OPENID_PRIVATE_KEY_PATH, OPENID_PRIVATE_KEY_PATH add scope, redirect_url to AuthorizationCode and OauthToken take into scope when creating oauth-token, authorization-code add jwcrypto add jose_utils: make_id_token and verify_id_token add &scope to redirect uri add "email_verified": True into user_info fix user not activated add /oauth2 as alias for /oauth handle case where scope and state are empty remove threaded=False Use Email Alias as wording remove help text user can re-send activation email add "expired" into ActivationCode Handle the case activation code is expired reformat: use form.validate_on_submit instead of request.method == post && form.validate use error text instead of flash() display client oauth-id and oauth-secret on client detail page not display oauth-secret on client listing fix expiration check improve page title, footer add /jwks and /.well-known/openid-configuration init properly tests, fix blueprint conflict bug in flask-admin create oauth_models module rename Scope -> ScopeE to distinguish with Scope DB model set app.url_map.strict_slashes = False use ScopeE instead of SCOPE_NAME, ... support access_token passed as args in /userinfo merge /allow-deny into /authorize improve wording take into account the case response_type=code and openid is in scope take into account response_type=id_token, id_token token, id_token code make sure to use in-memory db in test fix scope can be null allow cross_origin for /.well-known/openid-configuration and /jwks fix footer link center authorize form rename trial_expiration to plan_expiration move stripe init to create_app() use real email to be able to receive email notification add user.profile_picture_id column use user profile picture and fallback to gravatar use nguyenkims+local@gm to distinguish with staging handle plan cancel, reactivation, user profile update fix can_create_new_email create cron.py that set plan to free when expired add crontab.yml add yacron use notify_admin instead of LOG.error add ResetPasswordCode model user can change password in setting increase display time for notie add forgot_password page If login error: redirect to this page upon success login. hide discover tab add column user.is_developer only show developer menu to developer comment out the publish button set local user to developer make sure only developer can access /developer blueprint User is invited to upgrade if they are in free plan or their trial ends soon not sending email when in local mode create Partner model create become partner page use normal error handling on local fix migration add "import sqlalchemy_utils" into migration template small refactoring on setting page handle promo code. TODO: add migration file add migration for user.promo_codes move email alias on top of apps in dashboard add introjs move encode_url to utils create GenEmail.create_new_gen_email create a first alias mail to show user how to use when they login show intro when user visits the website the first time fix register
2019-07-02 09:20:12 +02:00
```
Make sure to uncomment the `RESET_DB=true` to create the database locally.
2019-12-09 22:26:58 +01:00
Feel free to custom your `.env` file, it would be your default setting when developing locally. This file is ignored by git.
create BaseForm to enable CSRF register page redirect user to dashboard if they are logged in enable csrf for login page Set models more strict bootstrap developer page add helper method to ModelMixin, remove CRUDMixin display list of clients on developer index, add copy client-secret to clipboard using clipboardjs add toastr and use jquery non slim display a toast when user copies the client-secret create new client, generate client-id using unidecode client detail page: can edit client add delete client implement /oauth/authorize and /oauth/allow-deny implement /oauth/token add /oauth/user_info endpoint handle scopes: wip take into account scope: display scope, return user data according to scope create virtual-domain, gen email, client_user model WIP create authorize_nonlogin_user page user can choose to generate a new email no need to interfere with root logger log for before and after request if user has already allowed a client: generate a auth-code and redirect user to client get_user_info takes into account gen email display list of clients that have user has authorised use yk-client domain instead of localhost as cookie depends on the domain name use wtforms instead of flask_wtf Dockerfile delete virtual domain EMAIL_DOMAIN can come from env var bind to host 0.0.0.0 fix signup error: use session as default csrf_context rename yourkey to simplelogin add python-dotenv, ipython, sqlalchemy_utils create DB_URI, FLASK_SECRET. Load config from CONFIG file if exist add shortcuts to logging create shell add psycopg2 do not add local data in Dockerfile add drop_db into shell add shell.prepare_db() fix prepare_db setup sentry copy assets from tabler/dist add icon downloaded from https://commons.wikimedia.org/wiki/File:Simpleicons_Interface_key-tool-1.svg integrate tabler - login and register page add favicon template: default, header. Use gravatar for user avatar url use default template for dashboard, developer page use another icon add clipboard and notie prettify dashboard add notie css add fake gen email and client-user prettify list client page, use notie for toast add email, name scope to new client display client scope in client list prettify new-client, client-detail add sentry-sdk and blinker add arrow, add dt jinja filter, prettify logout, dashboard comment "last used" in dashboard for now prettify date display add copy email to clipboard to dashboard use "users" as table name for User as "user" is reserved key in postgres call prepare_db() when creating new db error page 400, 401, 403, 404 prettify authorize_login_user create already_authorize.html for user who has already authorized a client user can generate new email display all other generated emails add ENV variable, only reset DB when ENV=local fix: not return other users gen emails display nb users for each client refactor shell: remove prepare_db() add sendgrid add /favicon.ico route add new config: URL, SUPPORT_EMAIL, SENDGRID_API_KEY user needs to activate their account before login create copy button on dashboard client can have multiple redirect uris, in client detail can add/remove redirect-uri, use redirect_uri passed in /authorize refactor: move get_user_info into ClientUser model dashboard: display all apps, all generated emails add "id" into user_info add trigger email button invalidate the session at each new version by changing the secret centralize Client creation into Client.create_new user can enable/disable email forwarding setup auto dismiss alert: just add .alert-auto-dismiss move name down in register form add shell.add_real_data move blueprint template to its own package prettify authorize page for non-authenticated user update readme, return error if not redirect_uri add flask-wtf, use psycopg2-binary use flask-wtf FlaskForm instead of Form rename email -> email_utils add AWS_REGION, BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY to config add s3 module add File model, add Client.icon_id handle client icon update can create client with icon display client icon in client list page add Client.home_url take into account Client.home_url add boto3 register: ask name first only show "trigger test email" if email forwarding is enabled display gen email in alphabetical order, client in client.name alphabetical order better error page the modal does not get close when user clicks outside of modal add Client.published column discover page that displays all published Client add missing bootstrap.bundle.min.js.map developer can publish/unpublish their app in discover use notie for display flash message create hotmail account fix missing jquery add footer, add global jinja2 variable strengthen model: use nullable=False whenever possible, rename client_id to oauth_client_id, client_secret to oauth_client_secret add flask-migrate init migrate 1st migrate version fix rename client_id -> oauth_client_id prettify UI use flask_migrate.upgrade() instead of db.create_all() make sure requirejs.config is called for all page enable sentry for js, use uppercase for global jinja2 variables add flask-admin add User.is_admin column setup flask admin, only accessible to admin user fix migration: add server_default replace session[redirect_after_login] by "next" request args add pyproject.toml: ignore migrations/ in black add register waiting_activation_email page better email wording add pytest add get_host_name_and_scheme and tests example fail test fix test fix client-id display add flask-cors /user_info supports cors, add /me as /user_info synonym return client in /me support implicit flow no need to use with "app.app_context()" add watchtower to requirement add param ENABLE_CLOUDWATCH, CLOUDWATCH_LOG_GROUP, CLOUDWATCH_LOG_STREAM add cloudwatch logger if cloudwatch is enabled add 500 error page add help text for list of used client display list of app/website that an email has been used click on client name brings to client detail page create style.css to add additional style, append its url with the current sha1 to avoid cache POC on how to send email using postfix add sqlalchemy-utils use arrow instead of datetime add new params STRIPE_API, STRIPE_YEARLY_SKU, STRIPE_MONTHLY_PLAN show full error in local add plan, plan_expiration to User, need to create enum directly in migration script, cf https://github.com/sqlalchemy/alembic/issues/67 reformat all html files: use space instead of tab new user will have trial plan for 15 days add new param MAX_NB_EMAIL_FREE_PLAN only user with enough quota can create new email if user cannot create new gen email, pick randomly one from existing gen emails. Use flush instead of commit rename STRIPE_YEARLY_SKU -> STRIPE_YEARLY_PLAN open client page in discover in a new tab add stripe not logging /static call: disable flask logging, replace by after_request add param STRIPE_SECRET_KEY add 3 columns stripe_customer_id, stripe_card_token, stripe_subscription_id user can upgrade their pricing add setting page as coming-soon add GenEmail, ClientUser to admin ignore /admin/static logging add more fake data add ondelete="cascade" whenever possible rename plan_expiration -> trial_expiration reset migration: delete old migrations, create new one rename test_send_email -> poc_send_email to avoid the file being called by pytest add new param LYRA_ANALYTICS_ID, add lyra analytics add how to create new migration into readme add drift to base.html notify admin when new user signs up or pays subscription log exception in case of 500 use sendgrid to notify admin add alias /userinfo to user_info endpoint add change_password to shell add info on how payment is handled invite user to retry if card not working remove drift and add "contact us" link move poc_send_email into poc/ support getting client-id, client-secret from form-data in addition to basic auth client-id, client-secret is passed in form-data by passport-oauth2 for ex add jwtRS256 private and public key add jwk-jws-jwt poc add new param OPENID_PRIVATE_KEY_PATH, OPENID_PRIVATE_KEY_PATH add scope, redirect_url to AuthorizationCode and OauthToken take into scope when creating oauth-token, authorization-code add jwcrypto add jose_utils: make_id_token and verify_id_token add &scope to redirect uri add "email_verified": True into user_info fix user not activated add /oauth2 as alias for /oauth handle case where scope and state are empty remove threaded=False Use Email Alias as wording remove help text user can re-send activation email add "expired" into ActivationCode Handle the case activation code is expired reformat: use form.validate_on_submit instead of request.method == post && form.validate use error text instead of flash() display client oauth-id and oauth-secret on client detail page not display oauth-secret on client listing fix expiration check improve page title, footer add /jwks and /.well-known/openid-configuration init properly tests, fix blueprint conflict bug in flask-admin create oauth_models module rename Scope -> ScopeE to distinguish with Scope DB model set app.url_map.strict_slashes = False use ScopeE instead of SCOPE_NAME, ... support access_token passed as args in /userinfo merge /allow-deny into /authorize improve wording take into account the case response_type=code and openid is in scope take into account response_type=id_token, id_token token, id_token code make sure to use in-memory db in test fix scope can be null allow cross_origin for /.well-known/openid-configuration and /jwks fix footer link center authorize form rename trial_expiration to plan_expiration move stripe init to create_app() use real email to be able to receive email notification add user.profile_picture_id column use user profile picture and fallback to gravatar use nguyenkims+local@gm to distinguish with staging handle plan cancel, reactivation, user profile update fix can_create_new_email create cron.py that set plan to free when expired add crontab.yml add yacron use notify_admin instead of LOG.error add ResetPasswordCode model user can change password in setting increase display time for notie add forgot_password page If login error: redirect to this page upon success login. hide discover tab add column user.is_developer only show developer menu to developer comment out the publish button set local user to developer make sure only developer can access /developer blueprint User is invited to upgrade if they are in free plan or their trial ends soon not sending email when in local mode create Partner model create become partner page use normal error handling on local fix migration add "import sqlalchemy_utils" into migration template small refactoring on setting page handle promo code. TODO: add migration file add migration for user.promo_codes move email alias on top of apps in dashboard add introjs move encode_url to utils create GenEmail.create_new_gen_email create a first alias mail to show user how to use when they login show intro when user visits the website the first time fix register
2019-07-02 09:20:12 +02:00
You don't need all the parameters, for example, if you don't update images to s3, then
2019-12-09 22:26:58 +01:00
`BUCKET`, `AWS_ACCESS_KEY_ID` can be empty or if you don't use login with Github locally, `GITHUB_CLIENT_ID` doesn't have to be filled. The `.env.example` file contains minimal requirement so that if you run:
2019-12-04 00:48:30 +01:00
```
2019-12-09 22:26:58 +01:00
python3 server.py
```
2019-12-04 00:48:30 +01:00
2019-12-09 22:26:58 +01:00
then open http://localhost:7777, you should be able to login with the following account
2019-12-04 00:48:30 +01:00
2019-12-09 22:26:58 +01:00
```
john@wick.com / password
```
2019-12-04 00:48:30 +01:00
2019-12-19 21:19:16 +01:00
### API
For now the only API client is the Chrome/Firefox extension. This extension relies on `API Code` for authentication.
In every request, the extension sends
2019-12-19 21:19:16 +01:00
- the `API Code` is set in `Authentication` header. The check is done via the `verify_api_key` wrapper, implemented in `app/api/base.py`
- the current website `hostname` which is the website subdomain name + domain name. For ex, if user is on `http://dashboard.example.com/path1/path2?query`, the subdomain is `dashboard.example.com`. This information is important to know where an alias is used in order to proposer to user the same alias if they want to create on alias on the same website in the future. The `hostname` is passed in the request query `?hostname=`, see `app/api/views/alias_options.py` for an example.
Currently, the latest extension uses the two following endpoints :
2019-12-19 21:19:16 +01:00
- `/alias/options`: returns what to suggest to user when they open the extension.
2019-12-19 21:19:16 +01:00
```
GET /alias/options hostname?="www.groupon.com"
Response: a json with following structure. ? means optional field.
recommendation?:
alias: www_groupon_com@simplelogin.co
hostname: www.groupon.com
custom:
suggestion: groupon
suffix: [@my_domain.com, .abcde@simplelogin.co]
can_create_custom: true
existing:
[email1, email2, ...]
```
- `/alias/custom/new`: allows user to create a new custom alias.
2019-12-29 15:43:59 +01:00
To try out the endpoint, you can use the following command. The command uses [httpie](https://httpie.org).
Make sure to replace `{api_key}` by your API Key obtained on https://app.simplelogin.io/dashboard/api_key
```
http https://app.simplelogin.io/api/alias/options \
Authentication:{api_key} \
hostname==www.google.com
```
2019-12-19 21:19:16 +01:00
```
POST /alias/custom/new
prefix: www_groupon_com
suffix: @my_domain.com
Response:
201 -> OK {alias: "www_groupon_com@my_domain.com"}
409 -> duplicated
```
### Database migration
2019-12-04 00:48:30 +01:00
2019-12-19 21:19:16 +01:00
The database migration is handled by `alembic`
2019-12-04 00:48:30 +01:00
Whenever the model changes, a new migration has to be created
2019-12-19 21:19:16 +01:00
Set the database connection to use a current database (i.e. the one without the model changes you just made), for example, if you have a staging config at `~/config/simplelogin/staging.env`, you can do:
2019-12-19 21:19:16 +01:00
2019-12-27 11:49:33 +01:00
```bash
ln -sf ~/config/simplelogin/staging.env .env
```
2019-12-19 21:19:16 +01:00
Generate the migration script and make sure to review it before committing it. Sometimes (very rarely though), the migration generation can go wrong.
2019-12-19 21:19:16 +01:00
2019-12-27 11:49:33 +01:00
```bash
flask db migrate
```
In local the database creation in Sqlite doesn't use migration and uses directly `db.create_all()` (cf `fake_data()` method). This is because Sqlite doesn't handle well the migration. As sqlite is only used during development, the database is deleted and re-populated at each run.
2019-12-19 21:19:16 +01:00
2019-12-19 22:36:37 +01:00
### Code structure
The repo consists of the three following entry points:
2019-12-19 22:36:37 +01:00
- wsgi.py and server.py: the webapp.
- email_handler.py: the email handler.
- cron.py: the cronjob.
Here are the small sum-ups of the directory structures and their roles:
2019-12-19 22:36:37 +01:00
- app/: main Flask app. It is structured into different packages representing different features like oauth, api, dashboard, etc.
- local_data/: contains files to facilitate the local development. They are replaced during the deployment.
- migrations/: generated by flask-migrate. Edit these files will be only edited when you spot (very rare) errors on the database migration files.
2019-12-19 22:36:37 +01:00
- static/: files available at `/static` url.
- templates/: contains both html and email templates.
2019-12-19 22:36:37 +01:00
- tests/: tests. We don't really distinguish unit, functional or integration test. A test is simply here to make sure a feature works correctly.
2019-12-19 21:19:16 +01:00
2019-12-22 12:57:16 +01:00
The code is formatted using https://github.com/psf/black, to format the code, simply run
```
black
```
2019-12-19 21:19:16 +01:00
### OAuth flow
SL currently supports code and implicit flow.
#### Code flow
To trigger the code flow locally, you can go to the following url after running `python server.py`:
```
http://localhost:7777/oauth/authorize?client_id=client-id&state=123456&response_type=code&redirect_uri=http%3A%2F%2Flocalhost%3A7000%2Fcallback&state=random_string
```
You should see there the authorization page where user is asked for permission to share their data. Once user approves, user is redirected to this url with an `authorization code`: `http://localhost:7000/callback?state=123456&code=the_code`
Next, exchange the code to get the token with `{code}` replaced by the code obtained in previous step. The `http` tool used here is https://httpie.org
```
http -f -a client-id:client-secret http://localhost:7777/oauth/token grant_type=authorization_code code={code}
```
This should return an `access token` that allows to get user info via the following command. Again, `http` tool is used.
2019-12-19 21:19:16 +01:00
```
http http://localhost:7777/oauth/user_info 'Authorization:Bearer {token}'
```
#### Implicit flow
Similar to code flow, except for the the `access token` which we we get back with the redirection.
2019-12-19 21:19:16 +01:00
For implicit flow, the url is
```
http://localhost:7777/oauth/authorize?client_id=client-id&state=123456&response_type=token&redirect_uri=http%3A%2F%2Flocalhost%3A7000%2Fcallback&state=random_string
```
#### OpenID and OAuth2 response_type & scope
2019-12-19 21:19:16 +01:00
According to the sharing web blog titled [Diagrams of All The OpenID Connect Flows](https://medium.com/@darutk/diagrams-of-all-the-openid-connect-flows-6968e3990660), we should pay attention to:
2019-12-19 21:19:16 +01:00
- `response_type` can be either `code, token, id_token` or any combination of those attributes.
- `scope` might contain `openid`
2019-12-19 21:19:16 +01:00
Below are the potential combinations that are taken into account in SL until now:
2019-12-19 21:19:16 +01:00
```
response_type=code
scope:
with `openid` in scope, return `id_token` at /token: OK
without: OK
response_type=token
scope:
with and without `openid`, nothing to do: OK
response_type=id_token
return `id_token` in /authorization endpoint
response_type=id_token token
return `id_token` in addition to `access_token` in /authorization endpoint
response_type=id_token code
return `id_token` in addition to `authorization_code` in /authorization endpoint
```
2019-12-04 00:48:30 +01:00
create BaseForm to enable CSRF register page redirect user to dashboard if they are logged in enable csrf for login page Set models more strict bootstrap developer page add helper method to ModelMixin, remove CRUDMixin display list of clients on developer index, add copy client-secret to clipboard using clipboardjs add toastr and use jquery non slim display a toast when user copies the client-secret create new client, generate client-id using unidecode client detail page: can edit client add delete client implement /oauth/authorize and /oauth/allow-deny implement /oauth/token add /oauth/user_info endpoint handle scopes: wip take into account scope: display scope, return user data according to scope create virtual-domain, gen email, client_user model WIP create authorize_nonlogin_user page user can choose to generate a new email no need to interfere with root logger log for before and after request if user has already allowed a client: generate a auth-code and redirect user to client get_user_info takes into account gen email display list of clients that have user has authorised use yk-client domain instead of localhost as cookie depends on the domain name use wtforms instead of flask_wtf Dockerfile delete virtual domain EMAIL_DOMAIN can come from env var bind to host 0.0.0.0 fix signup error: use session as default csrf_context rename yourkey to simplelogin add python-dotenv, ipython, sqlalchemy_utils create DB_URI, FLASK_SECRET. Load config from CONFIG file if exist add shortcuts to logging create shell add psycopg2 do not add local data in Dockerfile add drop_db into shell add shell.prepare_db() fix prepare_db setup sentry copy assets from tabler/dist add icon downloaded from https://commons.wikimedia.org/wiki/File:Simpleicons_Interface_key-tool-1.svg integrate tabler - login and register page add favicon template: default, header. Use gravatar for user avatar url use default template for dashboard, developer page use another icon add clipboard and notie prettify dashboard add notie css add fake gen email and client-user prettify list client page, use notie for toast add email, name scope to new client display client scope in client list prettify new-client, client-detail add sentry-sdk and blinker add arrow, add dt jinja filter, prettify logout, dashboard comment "last used" in dashboard for now prettify date display add copy email to clipboard to dashboard use "users" as table name for User as "user" is reserved key in postgres call prepare_db() when creating new db error page 400, 401, 403, 404 prettify authorize_login_user create already_authorize.html for user who has already authorized a client user can generate new email display all other generated emails add ENV variable, only reset DB when ENV=local fix: not return other users gen emails display nb users for each client refactor shell: remove prepare_db() add sendgrid add /favicon.ico route add new config: URL, SUPPORT_EMAIL, SENDGRID_API_KEY user needs to activate their account before login create copy button on dashboard client can have multiple redirect uris, in client detail can add/remove redirect-uri, use redirect_uri passed in /authorize refactor: move get_user_info into ClientUser model dashboard: display all apps, all generated emails add "id" into user_info add trigger email button invalidate the session at each new version by changing the secret centralize Client creation into Client.create_new user can enable/disable email forwarding setup auto dismiss alert: just add .alert-auto-dismiss move name down in register form add shell.add_real_data move blueprint template to its own package prettify authorize page for non-authenticated user update readme, return error if not redirect_uri add flask-wtf, use psycopg2-binary use flask-wtf FlaskForm instead of Form rename email -> email_utils add AWS_REGION, BUCKET, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY to config add s3 module add File model, add Client.icon_id handle client icon update can create client with icon display client icon in client list page add Client.home_url take into account Client.home_url add boto3 register: ask name first only show "trigger test email" if email forwarding is enabled display gen email in alphabetical order, client in client.name alphabetical order better error page the modal does not get close when user clicks outside of modal add Client.published column discover page that displays all published Client add missing bootstrap.bundle.min.js.map developer can publish/unpublish their app in discover use notie for display flash message create hotmail account fix missing jquery add footer, add global jinja2 variable strengthen model: use nullable=False whenever possible, rename client_id to oauth_client_id, client_secret to oauth_client_secret add flask-migrate init migrate 1st migrate version fix rename client_id -> oauth_client_id prettify UI use flask_migrate.upgrade() instead of db.create_all() make sure requirejs.config is called for all page enable sentry for js, use uppercase for global jinja2 variables add flask-admin add User.is_admin column setup flask admin, only accessible to admin user fix migration: add server_default replace session[redirect_after_login] by "next" request args add pyproject.toml: ignore migrations/ in black add register waiting_activation_email page better email wording add pytest add get_host_name_and_scheme and tests example fail test fix test fix client-id display add flask-cors /user_info supports cors, add /me as /user_info synonym return client in /me support implicit flow no need to use with "app.app_context()" add watchtower to requirement add param ENABLE_CLOUDWATCH, CLOUDWATCH_LOG_GROUP, CLOUDWATCH_LOG_STREAM add cloudwatch logger if cloudwatch is enabled add 500 error page add help text for list of used client display list of app/website that an email has been used click on client name brings to client detail page create style.css to add additional style, append its url with the current sha1 to avoid cache POC on how to send email using postfix add sqlalchemy-utils use arrow instead of datetime add new params STRIPE_API, STRIPE_YEARLY_SKU, STRIPE_MONTHLY_PLAN show full error in local add plan, plan_expiration to User, need to create enum directly in migration script, cf https://github.com/sqlalchemy/alembic/issues/67 reformat all html files: use space instead of tab new user will have trial plan for 15 days add new param MAX_NB_EMAIL_FREE_PLAN only user with enough quota can create new email if user cannot create new gen email, pick randomly one from existing gen emails. Use flush instead of commit rename STRIPE_YEARLY_SKU -> STRIPE_YEARLY_PLAN open client page in discover in a new tab add stripe not logging /static call: disable flask logging, replace by after_request add param STRIPE_SECRET_KEY add 3 columns stripe_customer_id, stripe_card_token, stripe_subscription_id user can upgrade their pricing add setting page as coming-soon add GenEmail, ClientUser to admin ignore /admin/static logging add more fake data add ondelete="cascade" whenever possible rename plan_expiration -> trial_expiration reset migration: delete old migrations, create new one rename test_send_email -> poc_send_email to avoid the file being called by pytest add new param LYRA_ANALYTICS_ID, add lyra analytics add how to create new migration into readme add drift to base.html notify admin when new user signs up or pays subscription log exception in case of 500 use sendgrid to notify admin add alias /userinfo to user_info endpoint add change_password to shell add info on how payment is handled invite user to retry if card not working remove drift and add "contact us" link move poc_send_email into poc/ support getting client-id, client-secret from form-data in addition to basic auth client-id, client-secret is passed in form-data by passport-oauth2 for ex add jwtRS256 private and public key add jwk-jws-jwt poc add new param OPENID_PRIVATE_KEY_PATH, OPENID_PRIVATE_KEY_PATH add scope, redirect_url to AuthorizationCode and OauthToken take into scope when creating oauth-token, authorization-code add jwcrypto add jose_utils: make_id_token and verify_id_token add &scope to redirect uri add "email_verified": True into user_info fix user not activated add /oauth2 as alias for /oauth handle case where scope and state are empty remove threaded=False Use Email Alias as wording remove help text user can re-send activation email add "expired" into ActivationCode Handle the case activation code is expired reformat: use form.validate_on_submit instead of request.method == post && form.validate use error text instead of flash() display client oauth-id and oauth-secret on client detail page not display oauth-secret on client listing fix expiration check improve page title, footer add /jwks and /.well-known/openid-configuration init properly tests, fix blueprint conflict bug in flask-admin create oauth_models module rename Scope -> ScopeE to distinguish with Scope DB model set app.url_map.strict_slashes = False use ScopeE instead of SCOPE_NAME, ... support access_token passed as args in /userinfo merge /allow-deny into /authorize improve wording take into account the case response_type=code and openid is in scope take into account response_type=id_token, id_token token, id_token code make sure to use in-memory db in test fix scope can be null allow cross_origin for /.well-known/openid-configuration and /jwks fix footer link center authorize form rename trial_expiration to plan_expiration move stripe init to create_app() use real email to be able to receive email notification add user.profile_picture_id column use user profile picture and fallback to gravatar use nguyenkims+local@gm to distinguish with staging handle plan cancel, reactivation, user profile update fix can_create_new_email create cron.py that set plan to free when expired add crontab.yml add yacron use notify_admin instead of LOG.error add ResetPasswordCode model user can change password in setting increase display time for notie add forgot_password page If login error: redirect to this page upon success login. hide discover tab add column user.is_developer only show developer menu to developer comment out the publish button set local user to developer make sure only developer can access /developer blueprint User is invited to upgrade if they are in free plan or their trial ends soon not sending email when in local mode create Partner model create become partner page use normal error handling on local fix migration add "import sqlalchemy_utils" into migration template small refactoring on setting page handle promo code. TODO: add migration file add migration for user.promo_codes move email alias on top of apps in dashboard add introjs move encode_url to utils create GenEmail.create_new_gen_email create a first alias mail to show user how to use when they login show intro when user visits the website the first time fix register
2019-07-02 09:20:12 +02:00