14 May 2021 · Software Engineering

    A CI/CD Pipeline for Serverless Cloudflare Workers

    9 min read
    Contents

    In this tutorial, we’ll learn how to use Semaphore to deploy serverless functions to Cloudflare. With serverless functions, developers can run production-ready applications that scale without having to manage infrastructure.

    Cloudflare Workers live on the edge of their network and can intercept and modify HTTP requests. Consequently, we can use them to augment our websites, create new applications and services, or redirect and load-balance traffic.

    As you transition from the “server full” to the serverless mindset, you’ll stop fretting about the how and start thinking in terms of the what. Your focus can move from managing infrastructure—whether they are servers or containers—to writing code. It’s on the cloud provider to figure out how to run it.

    In this post, we’ll learn how to use Semaphore CI/CD to test and deploy a serverless function to Cloudflare.

    Prerequisites

    You’ll need to set up a few things before reading on—after all, this is a hands-on tutorial.

    If you plan to test and develop on your machine, you should also install Node.js v10.16.2.

    Connect Semaphore with Cloudflare

    We need to share the Cloudflare account details with Semaphore, which entails generating an API Token:

    1. Go to Cloudflare and sign-in.
    2. Select your domain and go to the Overview tab.
    3. Scroll down to the API section. Copy the Zone ID and Account ID.
    4. Click on Get your API Key and copy the displayed email address.
    5. Go to the API Tokens tab.
    6. Click on the View button next to the Global API Key. Copy the authorization key.

    The token acts like a password, so we need to keep it off the repository. Semaphore provides secrets as a secure mechanism to store sensitive data.

    To create a secret from the Semaphore website, click on Secrets in the left navigation bar. Create the secret as shown with your Cloudflare account details:

    secret in semaphore for cloudflare
    Cloudflare Secret
    Secret details
    - Name: cloudflare
    - CLOUDFLARE_AUTH_EMAIL: YOUR_EMAIL
    - CLOUDFLARE_AUTH_KEY: YOUR_API_KEY
    - CLOUDFLARE_ACCOUNT_ID: YOUR_ACCOUNT_ID
    - CLOUDFLARE_ZONE_ID: YOUR_ZONE_ID

    Next, we’ll learn how to export the secrets in the job.

    Cloudflare Workers and Serverless

    In this section, we will learn how the serverless application works on Cloudflare.

    Begin by forking the repository; first, go to Semaphore demo and use the Fork button.

    Click on Clone or download to get the URL and clone it to your machine:

    $ git clone https://github.com/...

    Then, install the dependencies in your machine:

    $ npm install

    Finally, link your new repository with Semaphore:

    • Click on the + (plus) icon next to Projects:
    Add a project to Semaphore
    • Click Choose next to your repository:
    Select the repository to add.
    • Choose: I will use the existing configuration.

    Let’s take a few minutes to examine the code. The main function can be found at hello.js:

    addEventListener('fetch', event => {
      event.respondWith(handleRequest(event.request));
    });
    
    async function handleRequest(request) {
      return new Response('Hello World!');
    }

    Oh yes, here we have the archetypical “Hello, World example, only this time in the serverless form. The function replies “Hello World!” to any HTTP request, and that’s about it.

    To understand how serverless deployment works, we must zoom back and learn about the Serverless framework. Initially developed for AWS Lambda, the framework hides all the cloud provider quirks and boils down the description of the deployment into a single manifest file. Since its creation, many other providers have started supporting it.

    Open the manifest, serverless.yml to take a quick look. The service defines the service name and the provider-specific account details. Here is the definition for Cloudflare:

    service:
      name: semaphore-demo-cloudflare-workers
      config:
        accountId: $CLOUDFLARE_ACCOUNT_ID
        zoneId: $CLOUDFLARE_ZONE_ID
    
    provider:
      name: cloudflare
      stage: prod
    
    plugins:
      - serverless-cloudflare-workers

    Then, we need to define the functions that we want to call and the events that trigger them. In our example, requests to example.com/hello are replied with the response of the hello function (the .js extension can be omitted):

    functions:
      hello:
        name: hello
        worker: hello
        script: hello
        events:
          - http:
              url: example.com/hello
              method: GET

    Continuous Integration and Deployment

    For all its benefits, serverless has its downsides. Perhaps the biggest one is that testing gets more challenging. The cloud runtime is hard to replicate in a test environment. Nevertheless, we’ll try to simulate Cloudflare’s environs as best as we can with Semaphore Continuous Integration.

    Our pipelines will test the code on every update and then, provided there aren’t any errors, deploy it to Cloudflare. Here’s the complete workflow:

    Testing and deploying to Cloudflare

    We’ll learn about the testing pipeline next.

    Testing in Semaphore

    Semaphore will pick up existing CI/CD pipelines on the first push after adding the project:

    • Open a browser window to your GitHub repository.
    • Click Add new file to create an empty file.
    Create an empty file to trigger an initial CI/CD
    • Type any name for the file and Commit new file.
    • On your Semaphore account, you’ll find that the CI/CD process has already started:
    First run
    • Click on the running workflow:
    The CI/CD Pipelines in progress

    After the CI/CD process is done, the function should be deployed and ready to use in Cloudflare.

    Let’s examine how Semaphore works.

    To open the Workflow Builder, click on Edit the workflow.

    Opening the Workflow Builder

    The Workflow Builder main components are:

    • Agent: the agent is the combination of virtual machine type and operating system. We have some machines types to choose from, which, combined with an Ubuntu 18.04 image, make a complete platform to power the pipelines.
    • Pipelines: A pipeline determines the execution flow. A pipeline consists of a series of blocks that run from left to right. Each pipeline usually has a specific purpose, such as testing, deployment, etc.
    • Blocks: blocks bundle jobs with similar goals and settings. Blocks can have one or more jobs. Once all the jobs in a block complete, the next block begins.
    • Jobs: Jobs contain a list of commands to execute. Jobs within a block run in parallel.
    • Promotions: promotions connect pipelines to create complex workflows.
    The Workflow Builder main components

    Install Block

    Click on the Install dependencies block to examine it:

    Install Block

    The block has a single job called npm install with the following commands:

    checkout
    nvm use
    node -v
    cache restore
    npm install
    cache store

    The job uses nvm to set the Node.JS version (as determined in .nvmrc) and npm install downloads and installs dependencies on the node_modules directory.

    Checkout clones the GitHub repository, while cache detects the project structure and figures out which files should be cached for future runs. We’ll use cache store and cache retrieve to share the npm modules between jobs.

    Tests Block

    Open the Run tests block:

    Run tests block

    The prologue is executed before each job; in our case, we’re running again checkout, cache and nvm as setup commands.

    checkout
    cache restore
    nvm use
    node -v

    The second block has two test jobs:

    • Lint: uses JSHint, a code quality tool, to scan the files for code smells, that is, messy code that can cause bugs or decrease readability. Unused variables, missing semicolons, and using globals are all examples of funky code.
    • Unit tests: to replicate the Cloudflare environment in Semaphore, we are using the Cloudworker project. This library makes the code behave as if it were running in the Cloudflare cloud.

    Deploying with Semaphore

    Few things feel better than fully-tested code. It fills me with determination to continue with the deployment. The Continuous Deployment pipeline starts immediately after the integration ends.

    You’ll need to customize serverless.yml a bit now:

    • Open the file and locate the url key.
    • Replace example.com with your Cloudflare domain and type the URL you wish to trigger the function with.
    • You may use wildcards such as example.com/* or *.example.com.

    Scroll to the right and click on Deploy promotion:

    Configure automatic promotion
    • Enable the automatic promotion setting. This will trigger the Deploy pipeline for all successful runs in the master branch.
    • Edit the Deploy block, which has a single job called Deploy to Cloudflare:
    Deploy block

    The block deploys the function to Cloudflare. It has a:

    • Prologue: a prologue to set up the files and modules for the job.
    • Secret: the cloudflare secret you created earlier is activated. This imports the environment variables into all the jobs in the block.
    • Job: a job that replaces the environment variables in serverless.yml using the envsubst tool and deploys the code to Cloudflare. npm run deploy, calls the Serverless CLI to start the deployment. Within 30 seconds, the function should be up and running at Cloudflare. The last command calls the function directly from the cloud to test that it’s working.

    Run the pipeline

    You can trigger the CI/CD workflow by making a push to GitHub or by clicking on Run the Workflow and the Start:

    Running the CI/CD workflow

    You can check the progress on Semaphore; after a few seconds, we should see the deployment block complete. Clicking on the last job in the workflow brings up the execution log:

    Serverless: Packaging service...
    Serverless: Excluding development dependencies...
    Serverless: Starting Serverless Cloudflare-Worker deployment.
    Serverless: Starting deployment
    Serverless: deploying script: hello
    Serverless: Finished deployment hello in 1.412 seconds
    Serverless: ✅  Script Deployed. Name: hello, Size: 0.19K
    Serverless: ✅  Routes Deployed 
    Serverless: Finished deployment in 18.061 seconds.
    Serverless: Finished Serverless Cloudflare-Worker deployment.
    exit code: 0 duration: 20s

    Let’s see if it’s really working. Try visiting the function URL with your browser or making a request with curl.

    Replace example.com/hello as appropriate:

    $ curl -w "\n" example.com/hello
    Hello World!

    Well, hello to you too, little function.

    Servers Have Their Days Counted

    You’ve deployed your first serverless function. We’ve learned about its strengths and weaknesses, and how to balance them with Semaphore CI/CD. Isn’t its simplicity a breath of fresh air? If you are like me, once you try the serverless route, you’ll wish you could use it for everything.

    Want more practice? Try browsing the Cloudflare gallery which has a lot of good templates to explore.

    Don’t miss these serverless tutorials:

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    mm
    Writen by:
    I picked up most of my skills during the years I worked at IBM. Was a DBA, developer, and cloud engineer for a time. After that, I went into freelancing, where I found the passion for writing. Now, I'm a full-time writer at Semaphore.