This is a how-to post showing how to run an email subscribers list for your personal website using Modal webhooks and Gmail API. The application functionality allows a blog maintainer to accept and manage email suscribers who will automatically receive notifications when the maintainer publishes a new post to their RSS feed. This Jekyll static site you’re reading has RSS support built-in, and most other static site generating software does as well.
Besides the RSS feed, this newsletter solution requires only two other things: a Gmail account and a Modal account. Both are free to setup, so do that before proceeding 🙂.
Once you have those three things, we’re ready to start!
1. Gmail API setup
To set up the application to send emails on my behalf, I followed this Python quickstart guide from the official Google docs. The only non-trivial part was setting up the OAuth app, so I’ll walk you through that in a bit of detail.
Adding a constent screen and Oauth scopes:
After enabling the Gmail API in your Google Cloud project (mine is called thundergolfer-dotcom) the quickstart guide linked above tells you to create an OAuth client ID. This is necessary, but before I could do that I need to create an OAuth app registration, specify the requisite Oauth scopes, and fill-in a form to create an OAuth consent screen. The form is pictured below.
This is a Yak-shave, as this application won’t ever use the consent form, but Google made me do it so I did it. I don’t pay for Google Workspaces with my personal Google Account so I needed to create an ‘External OAuth’ app. This is fine because Google will put the app in ‘testing mode’ and still allow me to authenticate my own email account. Deploying with an OAuth app in ‘testing mode’ works for stuff like this; I had to do the similar to how it works for my Spotify OAuth app integration.
Some of the form fields are irrelevant but required, such as adding links to your app website and adding a logo. I just linked to my blog, and added a picture of myself 🤷.
Create OAuth Client ID
After finishing the consent page go back to Menu menu > APIs & Services > Credentials and create an OAuth client ID (pictured above). This will allow you to download a JSON file with personal credentials, including a client_id and client_secret. This file will be used to go through the OAuth flow yourself and acquire a crucial refresh_token granting indefinite access to Gmail sending for your account.
To go through the OAuth flow, take my code (linked above) and run the create_refresh_token_and_test_creds() function locally. The end result of the process is a .json file written to local disk.
python3 -m email_subs.main create-refresh-token
Once you have the .json file containing the refresh_token, copy from it the following fields into a Modal Secret called gmail.
| JSON file field | 'gmail' Modal Secret key |
|---|---|
client_id | GMAIL_AUTH_CLIENT_ID |
client_secret | GMAIL_AUTH_CLIENT_SECRET |
refresh_token | GMAIL_AUTH_REFRESH_TOKEN |
Done! The Modal application will use this populated gmail secret to authenticate with the GMail API.
Put your Google Cloud app ‘in production’
This last step is quick and very important. If you don’t do it, your unexpiring refresh_token will actually expire in 7 days. Go to the OAuth Consent Screen tab and under “Publishing Status” change your app from being in “testing” mode to being “in production”.
Now, Google will then say your app needs verification, but you can ignore this. What matters is that a production app’s refresh tokens will not expire and so the Modal email subscribers app won’t break after seven days.
For background on this crucial step, check out this StackOverflow answer.
2. Modal app setup
While I would prefer to inline all the code for this solution into the blog post, it would be too much, so I will just provide a summary of its structure. The code is heavily documented so as to be beginner friendly, and shouldn’t require modification besides changing some configuration values according to the commented instructions within.
App structure
The subscribers application is comprised of three web endpoint handlers (subscribe, confirm, unsubscribe), a simple SQlite database to store subscribers and notification history, and a cronjob function that checks the RSS feed and sends emails with GMail’s API. The three endpoints initiate state-transitions for the email passed as query parameter:
These endpoints are implemented by a FastAPI app and live in the main.py module.
web_app = FastAPI()
...
@web_app.get("/subscribe")
def subscribe(email: str): ...
@web_app.get("/confirm")
def confirm(email: str, code: str): ...
@web_app.get("/unsubscribe")
def unsubscribe(email: str, code: str): ...
Datastore
As said before, these endpoints store and manipulate state using an SQLite database file that lives on a Modal persistent shared volume (“persistent” means the data remains even if the application stops, and “shared” means it can be access by multiple Modal Functions).
The code for the database and its simple ORM is in datastore.py. The web handler functions use it like so:
conn = datastore.get_db(DB_PATH)
store = datastore.Datastore(
conn=conn,
codegen_fn=lambda: str(uuid.uuid4()),
clock_fn=lambda: datetime.now(timezone.utc),
)
notifications = store.list_notifications() # Show sent newsletter email notifications
confirmed = store.confirm_sub(email=email, code=code) # Confirm a subscriber
unsubbed = store.unsub(email=email, code=code) # Unsubscribe a subscriber
Emailer
Our application needs to be able to send emails to subscribers, and this is implemented in the emailer.py module.
It merely defines an ‘emailer’ interface and a single implementation, GmailSender(EmailSender):
class EmailSender(Protocol):
def send(self, message: EmailMessage) -> None:
...
A FakeEmailer(EmailSender) implementation exists in the tests.
Cronjob function
Finally, we have the cron-scheduled Modal Function which uses the feedparser Python package to download and parse the configured RSS feed.
It then checks the fetched RSS entries for blog posts that haven’t been previously sent to subscribers, creates basic HTML emails using the email_copy.py module, and sends them to subscribers.
3. Signup web component setup
Your readers will not want to hit the Modal web endpoint with curl to do signup. There should be a familiar, friendly web component interface with a simple text box input. That’s what you see at the top of this post. All it does is accept a (valid) email address and pass that to the web endpoint’s /subscribe handler, which will process the subscription and send back a confirmation.
If you’re a frontend afficianado you might be want write this functionality yourself into your website, maybe in React, Vue.js, or Svelte.
As this website is boring old Jekyll, the web component I wrote and provide in the source code is a rudimentary HTML and vanilla Javascript component, with CSS style. The file is called subscribe.html. You should be able to just copy paste this into your site; just update the endpoint URL and maybe tweak the CSS a bit 👍.
Testing, and launch
I released the functionality on my site recently and now have four, count em, four (4) subscribers! Gmail’s SREs feel trouble in the air whenever I push a new blog post.
Here’s what the last sent email looked like. I published a new post on my website around 5PM on the day, and an hour or so later the Modal cronjob ran and delivered this email to each subscriber.
If you set this up and need a first subscriber to test things, hit me up on Twitter or Reddit and I’ll be happy to be a testcase!