Serverless is one of those concepts that in hindsight seems obvious and makes you wonder why no one came up with it earlier.

At its core, it involves breaking up the application into smaller functions, called lambdas, and distributing them in the cloud. Lambdas, by their very nature, scale; this is because they are only run when needed, with resources allocated on-demand. Also, since there are no wasted resources, they tend to be cheaper to run than a full-fledged server or even a container.

Servers also rear their ugly heads during project development. Apps must be tested somewhere—too often a spare machine running some musty integration product. Once more, we find the same answer to the questions of upkeep and scaling: use a serverless continuous integration and delivery platform.

In this article, we will use two fantastic platforms to test and deploy an application: ZEIT Now to build and host it and Semaphore to drive continuous integration and delivery pipelines.

Serverless takes many forms. Some developers write lambdas, others choose an all-in-one solution platform. We could pick among one of the many serverless frameworks that have appeared in recent years—most of them require some degree of re-training and have the risk of vendor lock-in. ZEIT Now deployments, however, are seamless. ZEIT does not ask us to learn a new framework to be serverless. We can deploy our existing applications without modifications.

In this article, we will learn to use Semaphore to continuously test and deploy our application to ZEIT Now.

Meet the Application

During this post, we will work with the Semaphore Demo app. Step by step, we’ll see how to get it online and how to make part of a continuous delivery process.

The project has a single API Endpoint that simply replies Hello World! to all requests. APIs are the way applications communicate with each other over the web, so mastering them is crucial.

The main file is located at api/hello.js. This is the function that sends the hello message:

module.exports = (request, response) => {
	response.status(200).send('Hello World!');
};

Strictly speaking, this is the whole application. You could deploy this file and call it a day. We’ll see that ZEIT Now does a lot of work for us. However, the developer’s life calls for two more things: a development environment and some kind of testing.

Setting Up Your Dev Environment

Before advancing further, you may need to install some tools on your machine. You should check if you have installed the Node.js and npm bundle:

$ node -v
$ npm -v

If you need to install them, instructions can be found at the Node.js website. I recommend that you install the 10.16.3 version for its Long Term Support.

To work with the code, you will also need Git. Check if you already have it installed:

$ git --version

To run the application in your machine, install the Now CLI:

$ npm install now -g

Lastly, it’s a good idea to get curl, the Swiss army knife of networking.

Testing Lambdas

Testing is vital in any development, not only to find bugs but also to validate that the code exactly does what we want. As an example, the demo includes a script in the test directory that validates the response from the function.

Perhaps the main downside of using lambdas is that it makes testing more challenging. This is unavoidable since the cloud platform does so much for us. Nevertheless, we’ll use Semaphore CI along with some libraries to simulate as best as possible how the function behaves once deployed.

For testing, we’ll use Node.js and two libraries:

  • Express: Express is a framework designed for building websites and APIs. Its popularity has made it an integral part of web development. Unlike other frameworks, Express doesn’t force us to adopt any particular design pattern. Instead, it only provides the bare minimum functionality, which can be extended with middleware.
  • Jest: a testing framework. Jest scans the project files and executes the tests scripts.

Fork and Run the Demo

To get your own copy of the project, fork the repository on GitHub:

  1. Create GitHub account.
  2. Go to the Semaphore Demo and click on the Fork button on the top right side.
  3. Get your repository URL by clicking on Clone or download
  4. Clone the repository to your machine: git clone https://github.com/ …

You can get the app built with:

$ npm install

To run the test script:

$ npm test

> semaphore-demo-zeit-now@1.0.0 test /home/tom/r/semaphore-demo-zeit-now
> NODE_ENV=test PORT=3000 jest --coverage --bail

 PASS  test/api.test.js
  GET /api/hello
    ✓ respond with "Hello World!" (20ms)

  console.log test/api.test.js:10
    Server running on port 3000.

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |      100 |      100 |      100 |      100 |                   |
 hello.js |      100 |      100 |      100 |      100 |                   |
----------|----------|----------|----------|----------|-------------------|
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.143s
Ran all test suites.

Jest prints the result of each script. In the end, Jest prints a coverage report which tells us whether any part of the code was not executed.

No problems found? So far, so good.

The only thing left to do is to get the app started. The Now CLI can run lambdas directly in your machine:

$ now dev
> Now CLI 16.1.2 dev (beta) — https://zeit.co/feedback/dev
> Ready! Available at http://0.0.0.0:3000

You can view the response from your server either with curl or by opening your browser and going to http://127.0.0.1:3000.

$ curl -w "\n" http://0.0.0.0:3000/api/hello
Hello World!

You got it working? Way to go!

About ZEIT Now

With ZEIT Now, we can make a global serverless deployment with just a few keystrokes (seriously, it only takes one command). The magic resides in their builders which take your existing application code and transforms it into lambda functions. Do you have a lot of static files? No problem. With minification and a smart built-in global CDN, they have you covered.

Internally, Now Lambdas work on top of AWS Lambdas; ZEIT Now also supports other cloud providers (for caching and routing). As we’ll see next, we won’t have to configure or set up any service in AWS to run our application.

In ZEIT Now, there are no servers to manage and no containers to build. It integrates nicely with any workflow and plays well with CI/CD platforms. With instant, automatic deployments, ZEIT Now is a perfect fit for our microservice app.

Apps in ZEIT Now are immutable, meaning they cannot be changed once deployed. Each newly published version gets a unique deployment URL. As it turns out, this clever approach allows us to roll back to any previous version at any time.

ZEIT Now addresses follow this format:

  • https://APP_NAME.USERNAME.now.sh: Public-facing URL that points to the latest version.
  • https://APP_NAME-UUID.now.sh: Deployment URL. UUID is a unique, automatically generated string for each release.

Deploy From Your Machine

Enough theory. Time to get this app online:

  1. Sign up in ZEIT Now.
  2. Connect your machine with ZEIT Now and follow the on-screen instructions:
$ now login

Try deploying the staging site:

$ now deploy -n semaphore-demo-zeit-now-staging

> Deploying ~/r/semaphore-demo-zeit-now under YOUR_NAME
> Using project semaphore-demo-zeit-now-staging
> https://semaphore-demo-zeit-now-staging-cf88t6oz1.now.sh [v2] [2s]
> Ready! Deployed to https://semaphore-demo-zeit-now-staging.YOUR_NAME.now.sh [in clipboard] [34s]

Now does all the heavy lifting:

  • Uploads the code to ZEIT Now.
  • Downloads packages, builds and deploys the app.
  • Assigns deployment and public URLs.

Check out your new server. The ZEIT Now dashboard shows all deployments:

Deployments

What’s All the Buzz About Continuous Integration

At this point in the post, you may be asking: I got the app online all right, aren’t we done yet? Not by a long shot, the best part is yet to come. Think about all the things we did to get to this point. The first time it is always exciting and fun, but doing it over, again and again, will get old fast. Wouldn’t it be great if we could automate away all of it? This is precisely the problem that continuous integration (CI) and continuous delivery (CD) solve. CI/CD takes care of all the testing, building, and deploying. And they do it as a reproducible, battle-hardened process.

CI/CD is even more vital when working on a team. Lots of hands code faster but have a higher chance of introducing conflicts. The surest way to mitigate this is by building and testing the app as frequently as possible. Then, once everything is working, we can make high-quality releases faster.

About Semaphore

Older continuous integration and delivery platforms, like the stand-alone version of Jenkins, encumber developers an infrastructure to manage. In Semaphore there is no back-end to maintain, no servers to install, or any Java versions to troubleshoot. We only define pipelines in a clean, easy-to-read format and do as many Git pushes as needed. Semaphore will silently provision everything to drive the pipeline, at any scale.

Before diving deep with Semaphore, we need to learn about a few key concepts. Semaphore pipelines are written in YAML a text format that is easy for both humans and computers to read. When working with YAML, the important thing is to pay attention to indentation as, in the same vein of Python, spaces do matter.

For our project, we only need to know about a few properties, if curious, you may also check the full pipeline spec:

  • Name

Pipelines have a name which is shown on Semaphore’s dashboard. We also have to set the config version number. Right now the stable version is “v1.0”:

version: v1.0
name: Build and test
  • Agent

Semaphore offers several machine types with different specs. The combination of machine and operating system is defined under agent property. Semaphore provides an Ubuntu 18.04 image that is just right for our needs:

agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804
  • Blocks and Jobs

Blocks and jobs define what to do at each step of the pipeline. Each block can have many jobs. Jobs contain the list of commands to execute. Within a block, jobs run concurrently; each one runs in a fully isolated virtual machine. Once all jobs in a block are done, the next block begins.

This is how a block with two jobs looks like:

blocks:
  - name: Block name
    task:
      jobs:
      - name: My Job
        commands:
          - command 1
          - command 2
      - name: Another Job
        commands:
          - command 1
          - command 2

  • Promotions

Promotions chain pipelines together to build complex workflows. Since they can be triggered by user-defined conditions, they can be used to gracefully manage failures or to make a release when the pipeline runs successfully:

promotions:
  - name: Deploy
    pipeline_file: deploy.yml
    auto_promote_on:
      - result: passed

Continuous Integration Pipeline

In this section, we’ll review how the Continuous Integration pipeline works. The CI pipeline builds and tests the code in the Semaphore environment.

Open the pipeline file located at .semaphore/semaphore.yml. We’ve already learned about the agent so we’ll skip ahead to the first block.

Block 1: Install Dependencies

The first block downloads and installs all the required packages:

  • Checkout clones the code from GitHub.
  • Node npm and nvm are used to set Node.js version and install the packages.
  • Cache is used to share node modules between jobs. The cache is smart enough to determine which files need to be stored and retrieved.
blocks:
  - name: Install dependencies
    task:
      jobs:
        - name: npm install and cache
          commands:
            - checkout
            - nvm use
            - cache restore
            - npm install
            - cache store

Block 2: Testing

Since each job lives in an isolated environment, the second block must repeat some of the commands and get the node_modules from the cache.

The test starts at npm test, which is just a convenient way of starting Jest.

  - name: Run tests
    task:
      jobs:
        - name: npm test
          commands:
            - checkout
            - nvm use
            - cache restore
            - npm test

Promotions

Two promotions branch out of the CI pipeline:

  • Deploy to production: automatically started once all tests are green for the master branch.
  • Deploy to staging: can be manually initiated from a Semaphore workflow on any branch.
promotions:
  - name: Deploy to production
    pipeline_file: deploy-production.yml
    auto_promote_on:
      - result: passed
         branch:
           - master

Putting it All Together

Now that we have all the pieces of the puzzle in place, you will see for yourself the power and convenience of CI/CD.

A Shared Secret

To connect Semaphore and ZEIT Now, we need to get a token:

  1. Log in on your ZEIT Now account
  2. Go to Settings
  3. Go to the Tokens tab.
  4. Click on the Create button.
  5. Enter a name for the token, maybe something descriptive like semaphore-zeit-now.

The token, being private information, does not belong in the repository. Semaphore has a secure mechanism to handle secrets:

  1. Create an account for Semaphore with your GitHub login.
  2. On the left navigation bar, under Configuration, click on Secrets.
  3. Hit the Create New Secret button.
  4. Create the secret, as shown below:
Semaphore Secret

Continuous Delivery Pipeline

We have two almost identical Continuous Delivery pipelines: staging and production. They only differ in the deployment name. We use this feature to have two separate URLs for staging and production.

version: v1.0
name: Deploy to production
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804
blocks:
  - name: Deploy to production 
    task:
      secrets:
        - name: now
      jobs:
      - name: Deploy to Zeit Now 
        commands:
          - checkout 
          - nvm use 
          - npm install now -g 
          - now --token $ZEIT_TOKEN -n semaphore-demo-zeit-now

The deployment itself couldn’t be easier:

  1. The token is decrypted and imported as $ZEITTOKEN.
  2. The now utility is installed.
  3. Now does the deployment.

Start the pipeline

This is where all our hard work pays off:

  1. Go to your Semaphore account.
  2. Follow the link in the sidebar to create a new project.
  3. Semaphore will show your GitHub repositories, click on Add Repository
  4. The pipeline will start as soon changes are pushed into repository:
$ touch any_file
$ git add any_file
$ git commit -m "start semaphore pipeline"
$ git push origin master

In a few seconds, you should have your lambda function deployed to production. Click on the Promote button under “Deploy to staging” to create the staging site.

CI/CD Workflow

Once all the blocks are done, you should have both sites online. Try the APIs:

$ curl -w "\n" https://semaphore-demo-zeit-now-staging.YOUR_NAME.now.sh/api/hello
Hello World!

$ curl -w "\n" https://semaphore-demo-zeit-now.YOUR_NAME.now.sh/api/hello 
Hello World!

You Did It!

Nice work! Once you taste the power of serverless architecture and CI/CD, you will want to use it in all your projects. I hope that what you learned today helps you to build better and smarter.

If this article got you interested in serverless, you should check these links: