Skip to content

Latest commit

 

History

History
327 lines (243 loc) · 11.2 KB

README.md

File metadata and controls

327 lines (243 loc) · 11.2 KB
Raw

SMTP Gateway Plugin for RabbitMQ

This plugin makes SMTP and AMQP 0-9-1 interoperate. It can

  • convert incoming emails to AMQP 0-9-1 messages
  • consume AMQP 0-9-1 messages and send out emails

Certain interoperability with other protocols, namely STOMP, can be achieved as well.

This implementation aims to replace the rabbitmq-smtp plugin. It is based on a more advanced gen_smtp rather than on erlang-smtp.

This plugin is moderately mature. The described functionality is fully implemented and has been used in production for a couple of years. Feedback from users and test suite contributions are encouraged.

Build Status

RabbitMQ and Erlang Compatibility

Note: As of version 1.1.0, this plugin has been tested with RabbitMQ v3.11.x versions and Erlang/OTP 25.x

Installation

Due to the dependency on the eiconv library, which has a C source code component, this plugin must be compiled from source. If you are planning to use RabbitMQ version 3.11.1:

git clone https://github.com/gotthardp/rabbitmq-email.git
cd rabbitmq-email
make
make tests # optional
make RABBITMQ_VERSION=v3.11.1 dist

Copy the following directories to your RabbitMQ plugins/ directory:

plugins/eiconv-1.0.0
plugins/gen_smtp-1.1.1
plugins/rabbitmq_email-1.0.1

Documentation

The mapping between SMTP and AMQP 0-9-1 works in both directions. Before we provide a specific configuration example, let's take a look at the conceptual mapping between messages in the two protocols.

AMQP 0-9-1 to SMTP Conversion Workflow

The adapter consumes a set of AMQP queues (e.g. email-out). Each queue is linked with a "default" domain name. When a message is consumed, its AMQP routing key is examined to determine the target SMTP address.

  • routing key that includes a domain part (i.e. the "@" character") is mapped directly to an SMTP address
  • routing key that includes the mailbox name only (i.e. without "@") is combined with the "default" domain name assigned to the queue

To send emails, you should bind these queues to your exchange and then publish a message to this exchange. For example:

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters(host='localhost'))
channel = connection.channel()
channel.exchange_declare(exchange='X', type='topic')
channel.queue_bind(exchange='X', queue='email-out', routing_key='#')

channel.basic_publish(exchange='X',
    routing_key='[email protected]',
    properties=pika.BasicProperties(
        content_type = 'text/plain',
        headers = {'Subject':'Greetings'}),
    body='Hello world!')
connection.close()

The message gets converted as shown in the table below. No content filtering is performed in this direction.

AMQP SMTP
From: noreply@
routing_key To
message_id Message-Id
content_type Content-Type
headers additional headers

SMTP to AMQP 0-9-1 Conversion Workflow

The adapter listens for incoming emails. When an email arrives at the adapter, its SMTP "To" address is examined to determine how it should be routed through the system. First, the address is split into a mailbox name and a domain part.

  • the domain part (e.g. "@rabbitmq.com") is used to map sender's address to a RabbitMQ virtual host and exchange name
  • the mailbox name is mapped to an AMQP 0-9-1 routing key

To receive the incoming emails, simply bind your queue(s) to this exchange. For example:

import smtplib
from email.mime.text import MIMEText

me = "[email protected]"
you = "[email protected]"

msg = MIMEText("Hello world!")
msg['From'] = me
msg['To'] = you
msg['Subject'] = 'Greetings'

s = smtplib.SMTP('localhost', 2525)
s.login("guest", "guest")
s.sendmail(me, [you], msg.as_string())
s.quit()

To catch emails sent to unknown recipients you may use an Alternate Exchange.

When server_auth is false the server accepts e-mails from any client. When server_auth is rabbitmq the clients need to provide a username and password that is checked against the rabbitmq server.

SMTP Body Extraction

The email_filter configuration option can be used to extract information from the email body. When enabled, the adapter will:

  • select only one part of multipart content depending on user priority;
  • remove extra space from text content.

The function is optional, to not extract anything and send the entire e-mail as <<"application/mime">> set:

{email_filter, false}

Otherwise the email_filter identifies a list of content-types that shall be preferred. For example:

  • Extract the text body or (when no text) the first binary attachement. This is the default behaviour.

    {email_filter, [
  {<<"text">>, <<"plain">>},
  {<<"text">>, undefined},
  {undefined, undefined}
]}

    Each 2-tuple represents content type/subtype. The atom undefined represents any content other than <<"multipart">>.

  • Extract the first binary attachement or (when no attachement) the text body.

    {email_filter, [
  {binary, undefined},
  {<<"text">>, <<"plain">>},
  {<<"text">>, undefined}
]}

    The atom binary represents any content other than <<"text">> and <<"multipart">>.

SMTP Headers Extraction

Depending on the email_headers option the message gets converted as shown in the table below. The adapter will pass to AMQP only selected MIME headers

{email_headers, ["subject", "from", "charset"]},
SMTP AMQP
From
To exchange, routing_key
Message-Id message_id
timestamp
Subject Subject
Content-Type content_type

RabbitMQ Configuration Example

Like with most plugins, this one needs a configuration section in the RabbitMQ config file. RabbitMQ Configuration guide for more details.

All keys used by this plugin are under the rabbitmq_email section (app). Key settings are:

  • rabbitmq_email.server_config defines SMTP server parameters (same as in gen_smtp)
  • rabbitmq_email.email_domains maps sender domains to RabbitMQ virtual hosts and exchanges
  • rabbitmq_email.email_queues maps virtual hosts and queues to outgoing email domains
  • rabbitmq_email.client_config configures SMTP client settings for outgoing email
  • rabbitmq_email.email_from and rabbitmq_email.client_sender configure the FROM header used in outgoing emails

For example:

{rabbitmq_email, [
    %% gen_smtp server parameters
    {server_config, [
        {port, 2525}, {protocol, tcp}, {domain, "example.com"}, {address,{0,0,0,0}}
    ]},

    %% how clients are authenticated; either 'false' or 'rabbitmq' (default)
    {server_auth, rabbitmq},

    %% whether STARTTLS shall be offered; either 'true' or 'false' (default)
    {server_starttls, true},

    %% maps inbound email domains to vhosts and exchanges: [{email-domain, {vhost, exchange}}, ...}
    {email_domains, [
        {<<"example.com">>, {<<"/">>, <<"email-in">>}}
    ]},

    %% outbound email queues: [{{vhost, queue}, email-domain}, ...]
    {email_queues, [
        {{<<"/">>, <<"email-out">>}, <<"example.com">>}
    ]},

    %% sender indicated in the From header
    {email_from, <<"noreply">>},

    %% sender indicated in the SMTP from
    {client_sender, "[email protected]"},

    %% gen_smtp client parameters
    %% see https://github.com/gen-smtp/gen_smtp#client-example
    {client_config, [
        {relay, "smtp.example.com"}
    ]}
    ...
]}

Postfix Integration

You may want to run a standard Postfix SMTP server on the same machine and forward to the RabbitMQ plug-in only some e-mail domains.

  • Edit /etc/postfix/main.cf

    • Add the domains to be processed by RabbitMQ to the relay_domains list

      relay_domains = $mydestination, example.com

    • Make sure the transport_maps file is enabled

      transport_maps = hash:/etc/postfix/transport

  • Add links to the plug-in to the /etc/postfix/transport file

    example.com smtp:mail.example.com:2525
.example.com smtp:mail.example.com:2525

Getting Help

In case the plugin doesn't seem to work as expected, please start a rabbitmq-users thread and provide a way to reproduce:

  • RabbitMQ version used
  • Plugin version used
  • Steps to reproduce
  • Server logs

If the email => message workflow is used, please provide the exact email used. Do not use the forward button or modify the content in any way. Original MIME headers and body are critically important in troubleshooting.

Building from Source

You can build and install it like any other plugin (see the plugin development guide).

This plugin requires an Erlang NIF, eiconv. It is built automatically, but since this is a NIF (native code) its module is not portable between platforms.

Change Log

  • 1.1.1 (Oct 21, 2022)
  • 1.1.0 (Oct 19, 2022)
    • Compatibility with recent RabbitMQ versions.
      • 3.11.x
      • 3.10.x
    • Update gen_smtp to 1.2.0
    • Add ability to set max message size in PR #54
  • 1.0.0 (Nov 23, 2021)
    • Compatibility with recent RabbitMQ versions.
      • 3.9.x
  • 0.1.0 (Dec 22, 2015)
    • Compatibility changes for RabbitMQ 3.6.x.
  • 0.0.2 (Nov 14, 2015)
    • Supports authentication using RabbitMQ database.
    • Payload filtering is now optional (Issue #7).
  • 0.0.1 (May 12, 2015) First release.

Copyright and Licensing

Copyright (c) 2014-2017 Petr Gotthard [email protected] Copyright (c) 2017 Pivotal Software, Inc. Copyright (c) 2021 VMware, Inc. or its affiliates. All rights reserved.

This package is subject to the Mozilla Public License Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://mozilla.org/MPL/2.0/.

Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License.