SwiftDrop, or smtp2swift, receives email messages over SMTP and writes them to a configured OpenStack Swift container where they await further processing by some other process.
It consists of:
- a docker image;
- with a postfix SMTP daemon;
- some glue to make Postfix runnable inside docker;
- and a before-queue filter that stores incoming messages in swift.
The messages end up in Maildir-format inside the cur/ path in
the configured Swift container.
Basic usage involves:
- running the docker image;
- sending a message/rfc822 email message over SMTP on port 25;
- seeing the message end up in the Swift container, named something like
cur/1567067375.M6337175P228.f87ce1e3553a,S=241, where the first digits are the delivery unixtime.
See run-docker.sh for a sample invocation.
Processing the delivered mail is beyond the scope of this project, but a suggested operation is:
- listing the files in
cur/; - taking the first -- using a key-based global lock to lock that file --
downloading it, and moving it (copy+delete) to
processing/; - doing the processing;
- then -- again, using a lock -- moving it from
processing/to one ofdone/,failed/orretry/; - and using a separate job to delete very old messages.
See swiftq-example.py for sample dequeueing.
This project is licensed under the terms of the GPLv3 license.
You can configure accounts for SwiftDrop using environment variables in
the form SWIFTDROP_<SECTION>_<OPTION>.
You must specify at least one account where SECTION=DEFAULT. The
configuration options can be specified using UPPERCASE.
For example, a configuration with one Swift account that distributes incoming mail over an acceptance and a production container, for three distinct e-mail addresses, could be configured as follows:
# Postfix configuration (read by start wrapper and/or confd) # - postfix hostname, prefer having it resolvable, also used in HELO MYHOSTNAME=swiftdrop.example.com # - swiftdrop destinations (see also SWIFTDROP_xxx_RECIPIENTS) RECIPIENTS='[ "[email protected]", "[email protected]", "[email protected]"]' # - forwards to other destinations (postmaster, bounces, ...) FORWARDS='{ "[email protected]": "[email protected]", "[email protected]": "[email protected]", "[email protected]": "[email protected]"}' [email protected] # Swiftdrop configuration read by confd [email protected],[email protected] SWIFTDROP_DEFAULT_CONTAINER=acceptance # Swiftdrop configuration as used by swiftclient.Connection SWIFTDROP_DEFAULT_AUTH_VERSION=3 SWIFTDROP_DEFAULT_AUTHURL=https://keystone-server/v3 SWIFTDROP_DEFAULT_USER=myuser SWIFTDROP_DEFAULT_KEY=xxx SWIFTDROP_DEFAULT_OS_OPTIONS_PROJECT_DOMAIN_NAME=project_domain SWIFTDROP_DEFAULT_OS_OPTIONS_PROJECT_NAME=project SWIFTDROP_DEFAULT_OS_OPTIONS_USER_DOMAIN_NAME=user_domain # Swiftdrop configuration for a secondary address; # note that values from DEFAULT are available here too, # and need to be overwritten with blanks if they conflict. # For sanity, the DEFAULT above is the catch-all, and this # one handles only the "production" address. [email protected] SWIFTDROP_PRODUCTION_CONTAINER=production ...
The confd binary will convert the environment to
/etc/swiftdrop.ini, among others, during startup. Yielding a config
that may look somewhat like this:
[DEFAULT]
recipients = [email protected],[email protected]
container = acceptance
auth_version = 3
authurl = https://keystone-server/v3
user = myuser
key = xxx
os_options_project_name = project
os_options_project_domain_name = project_domain
os_options_user_domain_name = user_domain
[PRODUCTION]
recipients = [email protected]
container = production
...- Docker + Stretch image
- Postfix (3+)
- Python3.5 (comes with postfix install)
- Config of Swift accounts, and mail destinations through ENV
- main.cf config, for relay of desired recipients to Swift
- master.cf config
- Check Swift account (auth) on startup
- Only
2xxincoming mail if it is actually uploaded to Swift (otherwise it sends:451 4.3.0 Error: queue file write error) by using the before-queue smtpd_proxy_filter - Describe if/how do we cope with duplicates (email message-id cannot be used as globablly unique value)
- Add example code to dequeue stored mail (if possible with minimal dependencies): see examples/swiftq-example.py
- Opportunistic TLS has been observed to work out of the box.
- Hosting at OSSO in ossoio-k8s + swift2019.
- Allow forwarding of postmaster@ and other names to elsewhere.
- Enlarge message limits to 50MB.
- Add basic SPF checks?
- Add DKIM checks?
- Add countermeasures against spam / floods / other malicious stuff.
- Add currently implemented cur/MAILDIR scheme in synopsis at the top.
- Check/fix that SSL is kept up to date (both the ca-certificates -- for swift uploads -- and that postfix SSL snakeoil keys are kept up to date).
- Failures are now logged, but K8S is not scraped: do we need an ERRORMAIL (logging) handler?