At Rendered Text, we like to decompose our applications into microservices. These days, when we have an idea, we think of its implementation as a composition of multiple small, self-sustaining services, rather than an addition to a big monolith.
In a recent Semaphore product experiment, we wanted to create a service that gathers information from multiple sources, and then emails a report that summarizes the data in a useful way to our customers. It’s a good use case for a microservice, so let’s dive into how we did it.
Our microservices stack includes Elixir, RabbitMQ for communication, and Apache Thrift for message serialization.
We designed a mailing system that consists of three parts:
1. the main user-facing application that contains the data,
2. a data-processing microservice, and
3. a service for composing and sending the emails.
Asynchronous messaging with RabbitMQ
For asynchronous messaging, we decided to use RabbitMQ, a platform for sending and receiving messages.
First, we run the cron job inside our main application which gathers the data. Then, we encode that data and send it to the Elixir mailing microservice using RabbitMQ.
Every RabbitMQ communication pipeline consists of a producer and a consumer. For publishing messages from our main application, we have a Ruby producer that sends a message to a predefined channel and queue.
def self.publish(message)
options = {
:exchange => "exchange_name",
:routing_key => "routing_key",
:url => "amqp_url"
}
Tackle.publish(message, options)
end
In the last line above, we are using our open source tool called Tackle to publish a message. Tackle tackles the problem of processing asynchronous jobs in a reliable manner by relying on RabbitMQ. It serves as an abstraction around RabbitMQ’s API.
The consumer is a microservice written in Elixir, so we use ex-tackle, which is an Elixir port of Tackle:
elixir
defmodule MailingService.Consumer do
use Tackle.Consumer,
url: "amqp_url",
exchange: "exchange_name",
routing_key: "routing_key",
service: "service_name",
retry_limit: 10,
retry_delay: 10
def handle_message(message) do
message
|> Poison.decode!
|> Mail.compose
end
end
We connect to the specified exchange and wait for encoded messages to arrive. Options like retry_limit and retry_delay are there to allow us to specify how many times we want to retry message handling before the message is sent to the dead letter queue. The delay is there to set the timespan between each retry. This ensures the stability and reliability of our publish-subscribe system.
In our case, we use the decoded message to request data from the data processing microservice, and later use that response to send a message to the mailing service.
HTML template rendering in Elixir with EEx
Once our data is received and successfully decoded, we start composing the email by inserting the received data into the email templates. Similar to Ruby’s ERB and Java’s JSPs, Elixir has EEx, or Embedded Elixir. EEx allows us to embed and evaluate Elixir inside strings.
While using EEx, we go through three main phases. The first one is _evaluation_, the second is _definition_, and the third is _compilation_. EEx rules apply when a filename contains an extension html.eex.
Our email consists of multiple sections, all of which are using different datasets. Because of this, we divided our HTML email into partials for easier composing and improved code readability.
In order to evaluate the data inside a partial, we call the eval_file function and pass the data to partials:
elixir
<%%= EEx.eval_file "data/_introduction.html.eex",
[data1: template_data.data1,
data2: template_data.data2] %>
<%%= EEx.eval_file "data/_information.html.eex",
[data2: template_data.data2,
data3: template_data.data3] %>
Once we have all partials in place, we can combine them by evaluating them inside an entry point HTML template.
elixir
<%%= EEx.eval_file "data/_header.html.eex" %>
<%%= EEx.eval_file "data/_content.html.eex", [template_data: template_data] %>
Sending email with the SparkPost API
For email delivery, we rely on SparkPost. It provides email delivery services for apps, and it also provides useful email analytics. In our case, we used the Elixir SparkPost API client. We used it by creating a Mailer module that is very easy to use for email sending when instanced.
elixir
defmodule MailingService.Mailer do
alias SparkPost.{Content, Recipient, Transmission}
@return_path "semaphore+notifications@renderedtext.com"
def send_message(recipient_email, content) do
Transmission.send(%Transmission{
recipients: [ recipient_email ],
return_path: @return_path,
content: content,
campaign_id: "Campaign Name"
})
end
end
Once we’ve defined this module, we can easily use it anywhere as long as we pass the correct data structure. For example, we have a function that creates a data structure for the email template and passes it to the send_message function with desired recipients.
elixir
def compose_and_deliver(data1, data2) do
mail = %Content.Inline{
subject: "[Semaphore] #{TimeFormatter.today} Mail subject",
from: "Semaphore ",
text: template_data(data1, data2) |> text,
html: template_data(data1, data2) |> html
}
@mailer.send_message(data2["email"], mail)
end
SparkPost also enables us to send to multiple recipients at the same time, as well as send both HTML and plain text versions of an email. Bear in mind that you also need to provide .txt templates in order to send a plain text email.
As a final step in this iteration, we have developed a preview email that service owners receive a few hours before the production reports go out to customers.
Wrapping up
We now have a mailing microservice, made using Elixir, SparkPost, and RabbitMQ. Combining these three has allowed us to create a microservice that takes less than 4 seconds to gather data, send it, receive it on the other end, compose the emails, and dispatch them to customers.