A CI/CD Pipeline for Serverless Cloudflare Workers

In this tutorial, we’ll learn how to use Semaphore to deploy serverless functions to Cloudflare. With serverless functions, developers can run production-ready applications that scale without having to manage infrastructure.

Cloudflare Workers live on the edge of their network and can intercept and modify HTTP requests. Consequently, we can use them to augment our websites, create new applications and services, or redirect and load-balance traffic.

As you transition from the “server full” to the serverless mindset, you’ll stop fretting about the how and start thinking in terms of the what. Your focus can move from managing infrastructure—whether they are servers or containers—to writing code. It’s on the cloud provider to figure out how to run it.

In this post, we’ll learn how to use Semaphore CI/CD to test and deploy a serverless function to Cloudflare.

Prerequisites

You’ll need to set up a few things before reading on—after all, this is a hands-on tutorial.

• Install Git and sign up for a GitHub account.
• Sign up for Semaphore with your GitHub login.
• You’ll need a Cloudflare account with an active domain proxied through the service.

If you plan to test and develop on your machine, you should also install Node.js v10.16.2.

Connect Semaphore with Cloudflare

We need to share the Cloudflare account details with Semaphore, which entails generating an API Token:

1. Go to Cloudflare and sign-in.
2. Select your domain and go to the Overview tab.
3. Scroll down to the API section. Copy the Zone ID and Account ID.
4. Click on Get your API Key and copy the displayed email address.
5. Go to the API Tokens tab.
6. Click on the View button next to the Global API Key. Copy the authorization key.

The token acts like a password, so we need to keep it off the repository. Semaphore provides secrets as a secure mechanism to store sensitive data.

To create a secret from the Semaphore website, click on Secrets in the left navigation bar. Create the secret as shown with your Cloudflare account details:

Secret details
- Name: cloudflare
- CLOUDFLARE_AUTH_EMAIL: YOUR_EMAIL
- CLOUDFLARE_AUTH_KEY: YOUR_API_KEY
- CLOUDFLARE_ACCOUNT_ID: YOUR_ACCOUNT_ID
- CLOUDFLARE_ZONE_ID: YOUR_ZONE_ID

Next, we’ll learn how to export the secrets in the job.

Cloudflare Workers and Serverless

In this section, we will learn how the serverless application works on Cloudflare.

Begin by forking the repository; first, go to Semaphore demo and use the Fork button.

Click on Clone or download to get the URL and clone it to your machine:

$ git clone https://github.com/...

Then, install the dependencies in your machine:

$ npm install

Finally, link your new repository with Semaphore:

• Click on the + (plus) icon next to Projects:

• Click Choose next to your repository:

• Choose: I will use the existing configuration.

Let’s take a few minutes to examine the code. The main function can be found at hello.js:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request));
});

async function handleRequest(request) {
  return new Response('Hello World!');
}

Oh yes, here we have the archetypical “Hello, World“ example, only this time in the serverless form. The function replies “Hello World!” to any HTTP request, and that’s about it.

To understand how serverless deployment works, we must zoom back and learn about the Serverless framework. Initially developed for AWS Lambda, the framework hides all the cloud provider quirks and boils down the description of the deployment into a single manifest file. Since its creation, many other providers have started supporting it.

Open the manifest, serverless.yml to take a quick look. The service defines the service name and the provider-specific account details. Here is the definition for Cloudflare:

service:
  name: semaphore-demo-cloudflare-workers
  config:
    accountId: $CLOUDFLARE_ACCOUNT_ID
    zoneId: $CLOUDFLARE_ZONE_ID

provider:
  name: cloudflare
  stage: prod

plugins:
  - serverless-cloudflare-workers

Then, we need to define the functions that we want to call and the events that trigger them. In our example, requests to example.com/hello are replied with the response of the hello function (the .js extension can be omitted):

functions:
  hello:
    name: hello
    worker: hello
    script: hello
    events:
      - http:
          url: example.com/hello
          method: GET

Continuous Integration and Deployment

For all its benefits, serverless has its downsides. Perhaps the biggest one is that testing gets more challenging. The cloud runtime is hard to replicate in a test environment. Nevertheless, we’ll try to simulate Cloudflare’s environs as best as we can with Semaphore Continuous Integration.

Our pipelines will test the code on every update and then, provided there aren’t any errors, deploy it to Cloudflare. Here’s the complete workflow:

We’ll learn about the testing pipeline next.

Testing in Semaphore

Semaphore will pick up existing CI/CD pipelines on the first push after adding the project:

• Open a browser window to your GitHub repository.
• Click Add new file to create an empty file.

• Type any name for the file and Commit new file.
• On your Semaphore account, you’ll find that the CI/CD process has already started:

• Click on the running workflow:

After the CI/CD process is done, the function should be deployed and ready to use in Cloudflare.

Let’s examine how Semaphore works.

To open the Workflow Builder, click on Edit the workflow.

The Workflow Builder main components are:

• Agent: the agent is the combination of virtual machine type and operating system. We have some machines types to choose from, which, combined with an Ubuntu 18.04 image, make a complete platform to power the pipelines.
• Pipelines: A pipeline determines the execution flow. A pipeline consists of a series of blocks that run from left to right. Each pipeline usually has a specific purpose, such as testing, deployment, etc.
• Blocks: blocks bundle jobs with similar goals and settings. Blocks can have one or more jobs. Once all the jobs in a block complete, the next block begins.
• Jobs: Jobs contain a list of commands to execute. Jobs within a block run in parallel.
• Promotions: promotions connect pipelines to create complex workflows.

Install Block

Click on the Install dependencies block to examine it:

The block has a single job called npm install with the following commands:

checkout
nvm use
node -v
cache restore
npm install
cache store

The job uses nvm to set the Node.JS version (as determined in .nvmrc) and npm install downloads and installs dependencies on the node_modules directory.

Checkout clones the GitHub repository, while cache detects the project structure and figures out which files should be cached for future runs. We’ll use cache store and cache retrieve to share the npm modules between jobs.

Tests Block

Open the Run tests block:

The prologue is executed before each job; in our case, we’re running again checkout, cache and nvm as setup commands.

checkout
cache restore
nvm use
node -v

The second block has two test jobs:

• Lint: uses JSHint, a code quality tool, to scan the files for code smells, that is, messy code that can cause bugs or decrease readability. Unused variables, missing semicolons, and using globals are all examples of funky code.
• Unit tests: to replicate the Cloudflare environment in Semaphore, we are using the Cloudworker project. This library makes the code behave as if it were running in the Cloudflare cloud.

Deploying with Semaphore

Few things feel better than fully-tested code. It fills me with determination to continue with the deployment. The Continuous Deployment pipeline starts immediately after the integration ends.

You’ll need to customize serverless.yml a bit now:

• Open the file and locate the url key.
• Replace example.com with your Cloudflare domain and type the URL you wish to trigger the function with.
• You may use wildcards such as example.com/* or *.example.com.

Scroll to the right and click on Deploy promotion:

• Enable the automatic promotion setting. This will trigger the Deploy pipeline for all successful runs in the master branch.
• Edit the Deploy block, which has a single job called Deploy to Cloudflare:

The block deploys the function to Cloudflare. It has a:

• Prologue: a prologue to set up the files and modules for the job.
• Secret: the cloudflare secret you created earlier is activated. This imports the environment variables into all the jobs in the block.
• Job: a job that replaces the environment variables in serverless.yml using the envsubst tool and deploys the code to Cloudflare. npm run deploy, calls the Serverless CLI to start the deployment. Within 30 seconds, the function should be up and running at Cloudflare. The last command calls the function directly from the cloud to test that it’s working.

Run the pipeline

You can trigger the CI/CD workflow by making a push to GitHub or by clicking on Run the Workflow and the Start:

You can check the progress on Semaphore; after a few seconds, we should see the deployment block complete. Clicking on the last job in the workflow brings up the execution log:

Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Starting Serverless Cloudflare-Worker deployment.
Serverless: Starting deployment
Serverless: deploying script: hello
Serverless: Finished deployment hello in 1.412 seconds
Serverless: ✅  Script Deployed. Name: hello, Size: 0.19K
Serverless: ✅  Routes Deployed 
Serverless: Finished deployment in 18.061 seconds.
Serverless: Finished Serverless Cloudflare-Worker deployment.
exit code: 0 duration: 20s

Let’s see if it’s really working. Try visiting the function URL with your browser or making a request with curl.

Replace example.com/hello as appropriate:

$ curl -w "\n" example.com/hello
Hello World!

Well, hello to you too, little function.

Servers Have Their Days Counted

You’ve deployed your first serverless function. We’ve learned about its strengths and weaknesses, and how to balance them with Semaphore CI/CD. Isn’t its simplicity a breath of fresh air? If you are like me, once you try the serverless route, you’ll wish you could use it for everything.

Want more practice? Try browsing the Cloudflare gallery which has a lot of good templates to explore.

Don’t miss these serverless tutorials:

• Continuous Delivery Patterns with Serverless Applications
• Netlify Continuous Deployment