Semaphore Blog

News and updates from your friendly continuous integration and deployment service.

Continuous Delivery of Docker Images with Quay and Semaphore

Get future posts like this one in your inbox.

Follow us on

Running code in different environments can often produce unexpected behavior. Development and testing environments strive to replicate production envrionments as closely as possible, but there’s still much to be desired. This is where containers come in. When using Docker-based deployments, environments for development, staging and production are virtually the same, and they can be rapidly reproduced on-demand. As more and more developers find these benefits appealing, there’s an emerging need for hosting and managing Docker images without having an own infrastructure to do so.

Continuous Delivery of Docker Images with Quay and Semaphore

Quay.io is one of the first registry-as-a-service providers. It’s focused on covering all the aspects of hosting and managing Docker and rkt images, with seamless conversion between these formats. It has an intuitive interface, which gives in-depth insights into various stages of image and team management.

For performing security scans on images, Quay.io integrates with Clair by CoreOS. Detailed reports are provided after each scan, pinpointing the security vurnelabilities and their severity. Permission handling is also an important aspect of image management, and Quay.io provides fine grained controls over image access control, including robot accounts for deployments.

Image repositories hosted on Quay.io can be public or private. The number of public repositories is unlimited. Various paid plans are available for hosting private repositories, which also include team management features. Quay.io also offers an on-permise solution with Quay Enterprise.

Configuring the Quay Integration

By integrating with Quay, we wanted to make the process of accessing your private images more stranghtforward.

Projects using Semaphore’s Docker-enabled platform can configure this integration with the Quay project add-on. To set it up, visit Semaphore Add-ons on the project page, click on “Docker Registry”, and then select the Quay registry.

Docker Container Registry with Semaphore Add-ons

A form will be shown for filling out your credentials.

Quay Setup on Semaphore

After completing this step, the authentication details will be safely stored and ready to use. When running builds and deployments, you will be automatically logged into Quay.io, giving you access to all of your images.

Quay Container Registry Deployment

For a detailed overview of your Docker images, visit the Quay.io dashboard.

Quay Container Registry Dashboard

To start building your Docker project on Semaphore using the Quay integration, create a free Semaphore account and add your Docker project.

We hope that our Quay integration will make the continuous delivery of your containerized applications more convenient. Let us know how you like it in the comments below or by contacting us on support.

Happy building! 🐳

Announcing Preview of Semaphore Enterprise Continuous Integration and Delivery

Today, we’re happy to introduce a preview of Semaphore Enterprise, a new edition of Semaphore that helps large companies take advantage of our continuous integration and delivery (CI/CD) solution, while meeting their company security standards and need for advanced support and access control. We’re also proud to announce a partnership with Docker, who will make Semaphore Enterprise available on their brand new Docker Store. Request an invite to get started.

Semaphore Enterprise Continuous Integration and Delivery

Key features

On-premise CI and CD combined with an easy-to-use web UI

Semaphore Enterprise combines the power of on-premise continuous integration and delivery with the convenience of using Semaphore’s hosted, easy-to-use user interface. With Semaphore Enterprise, your code will never leave your premises, giving you full control over the security, performance and capacity of your CI environment.

Packaged in a certified Docker container

The core component of Semaphore Enterprise is the Job Runner, packaged in a certified Docker container, which you can download from Docker Store. Announced at DockerCon in Seattle, Docker Store is currently available in limited beta. Its general availability is planned for the third quarter of this year. You can follow our blog for more information about the public launch.

Works with your preferred environment and tools

Since it runs in a Docker container, the Job Runner is simple to run and operate. Best of all, being able to run it on any machine means that Semaphore Enterprise works with any technology and on any operating system that can run Docker.

Simple to set up and use

You can run as many Job Runners as you need, and Semaphore will do the rest: distribute and coordinate jobs, send notifications, manage users and permissions, and provide API endpoints.

First-class support

One of the aspects of Semaphore we take most pride in is the support we provide to our users. From setup to release, our experienced team of developers become the extended members of your development team when needed. This gives enterprises the confidence needed to practice continuous integration and delivery even on their most complex applications.

Getting started

Semaphore Enterprise is a culmination of the experience we have gained developing one of the most popular hosted CI services, known for its ease of use, reliability and great technical support. It is currently available to try for free as a preview release. Pricing will be announced in the upcoming weeks.

We’re excited to bring this new product to you, and we’d love to hear what you think! Please feel free to post your questions or feedback in the comments below, or contact us directly on support. If you’d like to try out Semaphore Enterprise, request an invite to get started.

Next steps

Want to learn more?

Happy building! 🚀

Continuous Delivery of Docker Images with Google Container Registry and Semaphore

As more and more developers see the benefit in delivering their apps through containers, managing Docker images is becoming increasingly important. Google Container Registry (GCR) provides a way to handle this need in a simple and secure way.

Continuous Delivery of Docker Images with Google Container Registry and Semaphore

GCR runs on the Google Cloud Platform, leveraging its availability, performance and cost effectiveness. In addition to this, Google Container Engine (GKE) can run your images in a highly scalable fashion. To achieve this, a group of Compute Engine instances run Kubernetes for orchestration. Different scaling strategies can be set up and triggered by predefined usage levels of resources such as CPU and memory.

Google’s regional repositories enable fast image pull times, selecting the optimal source based on geographical location. GCR can also take advantage of Cloud Storage Nearline buckets for archival storage. Container security is important and GCR handles it by integrating with Twistlock for image scanning, access control functions and enforcing runtime security policies.

Google has been using containers to run their backend for a decade, and GKR and GCR are a great way to leverage their experience for your own contaier needs.

Configuring the Google Container Registry Integration

Docker images need to be cushioned with secure and fast infrastructure to ensure smooth delivery. Semaphore’s integraton with Google Container Registry aims to do just that. Semaphore uses GCR’s advanced authentication to automatically give you access to your private images during builds and deployments.

Projects using Semaphore’s Docker-enabled platform can configure this integration with the Google Container Registry project add-on. To set it up, visit Semaphore Add-ons on the project page, click on “Docker Registry”, and then select the Google Container Registry.

Docker Container Registry with Semaphore Add-ons

This will bring up a form where you can fill out your credentials.

Google Container Registry Setup on Semaphore

After saving the credentials, they will be safely stored and ready to use. During builds and deployments, you will automatically have access to all of your private images in the GCR.

Google Container Registry Deployment

For an overview of your deployed images, you can visit the Google Container Registry dashboard.

Google Container Registry Hub

To start building your Docker project on Semaphore using our new Google Container Registry integration, you can create a free Semaphore account and add your Docker project.

We’re excited to bring Semaphore’s new GCR integration to you, and we hope that it will help with making the continuous delivery of your dockerized applications simpler and faster. We would love to hear what you think in the comments below or through our support channels.

Happy building! 🐳

Continuous Delivery of Docker Images with Docker Hub and Semaphore

With the rapidly growing use of microservices, fast and scalable image distribution has become a critical aspect of the software development practice. If you’re using Docker for developing your applications, you’re already familiar with container registries such as Docker’s own Docker Hub. Container registries are used to store, fetch, or deploy Docker images quickly and reliably.

Docker Hub is a cloud-based image registry, covering all aspects of image management, starting from secure storage, team management, and official repositories to running checks for security vulnerabilities.

Continuous Delivery of Docker Images with Docker Hub and Semaphore

Docker Hub repositories can either be public, allowing you to share your images with the Docker community, or private, providing teams with fine-grained control over user permissions. Images can originate from a Docker daemon running locally, from a CI server, or alternatively you can use Docker Hub’s automated builds feature. Shared images can be discovered through Docker Hub, or searched directly from your terminal through the CLI. Official repositories are offered by various providers like Ubuntu, Postgres, MySQL and others, ensuring that security patches are available as soon as possible. Official repositories also serve the purpose of promoting best practices for Dockerfiles, as well as minimizing the overhead when setting up environments for various programming languages or databases.

Team management is also streamlined with organizations and collaborators, who can have various permissions levels for accessing the repository and managing images.

Configuring the Docker Hub Integration

Since the use of container registries is rapidly spreading, we wanted to ensure that accessing them from Semaphore is effortless. Our new Docker Hub integration allows you to securely store your credentials, and avoid having to manage the authentication process with commands. Everything is handled automatically.

Projects using Semaphore’s Docker-enabled platform can configure this integration with the Docker Hub project add-on. To set it up, visit Semaphore Add-ons on the project page, click on “Docker Registry”, and then select the Docker Hub registry.

Docker Container Registry with Semaphore Add-ons

This will bring up a form where you can fill out your credentials.

Docker Hub Setup on Semaphore

After saving the credentials, they will be safely stored and ready to use. During builds and deployments, you will be automatically authenticated with Docker Hub, which will give you access to all of your images in the registry.

Docker Hub Container Registry Deployment

You can visit Docker Hub to get a detailed overview of your Docker images.

Docker Hub Container Registry Dashboard

To start building your Docker project on Semaphore using our new Docker Hub integration, you can create a free Semaphore account and add your Docker project.

We hope that Semaphore’s new Docker Hub integration will help streamline the continuous delivery of your dockerized applications. Let us know how you like it in the comments below or by contacting us on support.

Happy building! 🐳

Platform Update on June 21st

The upcoming platform update is scheduled for June 21st, 2016.

Chromdriver gets an update with version 2.22.

Elixir receives an update with version 1.2.6.

Git has been updated to version 2.9.0, introducing parallel cloning and updating of submodules.

MySQL receives an update with version 5.6.31.

Node.js gets an update with version 4.4.5.

PHP gets three version updates with 5.5.36, 5.6.22 and 7.0.7.

RabbitMQ has been updated to version 3.6.2.

RethinkDB gets an update with version 2.3.4.

New things

The following additions will be available after switching to the release candidate platform.

Node.js 6.2.1 is now part of the platform. To use it, add nvm use 6.2 to your setup commands.

Qt 5.5 has been added to the platform. It introduces a suite of new features, deprecates Qt WebKit and utilizes the Chromium based Qt WebEngine. Adding QT_SELECT=55 as an environment variable will activate this version. Alternatively, you can prepend one of your build commands with the variable above, so that Qt 5.5 is used only during its execution.

Trying the new platform

To ensure that the updates are compatible with your current setup, please switch to the Ubuntu 14.04 LTS v1606 (release candidate) platform in Project Settings > Platform. We’re looking forward to hearing your feedback and requirements, which will help us to fix the potential issues and tailor the platform to better suit your needs. The release candidate period will last until June 21st, 2016.

Changes in the final release

Qt 5.5 now includes the qt55webkit package as well.

The Docker-enabled platform gets an update with docker-engine 1.11.2.

A full list of changes is available in the platform changelog.

Continuous Delivery of Docker Images with Amazon ECR and Semaphore

Docker’s increasing popularity and the move toward microservices grew the need for container registries such as the Amazon EC2 Container Registry (ECR). Cloud-based container registries provide an easy way to store, retrieve or deploy Docker images in a fast and scalable way.

Amazon ECR is one of the most popular services for storing and keeping track of private container images. ECR solves the problem of having to install, operate and scale the infrastructure for a self-hosted container registry.

Continuous Delivery of Docker Images with Amazon ECR and Semaphore

AWS is a popular option for hosting applications in the cloud, and a lot developers moving to Docker are now also using Amazon ECR for their dockerized applications. Same as for other Amazon services, the pricing is based on the amount of data stored and transferred, minimizing the cost of running the infrastructure. ECR is also attractive to developers that are already using the Elastic Container Service, as it simplifies controlling user access and managing image permissions through the AWS Identity and Access Management (IAM) system.

Configuring the Amazon ECR Integration

With the growing use of private container registries in mind, we wanted to make the process of handling your Docker images more straightforward. Manually managing your credentials during builds and deployments is cumbersome and it can disrupt your focus, so we decided to handle it on our side.

Our new Amazon EC2 Container Registry integration allows you to store your credentials on Semaphore, and skip having to do this manually when you’re setting up your commands.

This integration can be configured for any project which uses Semaphore’s Docker-enabled platform. To set it up, visit Semaphore Add-ons on the project page, click on “Docker Registry”, and then select the Amazon EC2 Container Registry.

Docker Container Registry with Semaphore Add-ons

This will bring up a form where you can enter your credentials.

Docker Container Registry List

After saving the credentials, they will be verified and securely stored.

Once the credentials are configured, each time you build and deploy, you will be automatically logged into the Amazon EC2 Container Registry service, which will allow you to push and pull your private images.

AWS EC2 Container Registry Deployment

To get a detailed overview of your Docker images, visit the Amazon EC2 Container Service Dashboard.

AWS EC2 Container Registry Dashboard

If you are ready to start building your Docker project on Semaphore using our new Amazon ECR integration, you can create a free account and add your Docker project to get started.

We are excited to share this integration with you, and hope that it will help simplify your container management workflow. Let us know how you like it in the comments or by contacting us on support.

Happy building! 🐳

Continuous Integration and Deployment for Docker

This week, we are more than thrilled to announce that Semaphore now officialy supports Docker. The service update, immediately available to all Semaphore users, brings full technical capabilities of using Docker and additions to the user interface with the goal of simplifying your continuous integration and delivery workflow.

Docker Continuous Delivery with Semaphore

Docker: Package and Deploy Your Application Anywhere

A Docker container is similar to a virtual machine, but, unlike a virtual machine, it allows the contained application to use the same kernel as the system it is running on. This model brings software isolation at high performance and very low footprint, thus making it faster and easier to create and deploy.

Once your application and its dependencies are in a Docker container, you can deploy, scale and run that container on any platform or machine that supports Docker. Using a standardized package format guarantees that your application and its dependencies will always run the same in any environment.

These technical capabilities have made Docker interesting both to enterprises that want to run their own PaaS, as well as teams that want to develop microservices using various programming languages and frameworks, and deploy them continuously, without having to worry about the complexities of maintaining the supporting software stack in production.

Finally, while Docker has started its life on Linux, it has already made leaps towards being capable of running on Windows and Mac as well.

Efficient Continuous Delivery for Docker

Semaphore’s Docker-enabled platform has been in open beta for quite some time now. We gathered a lot of useful feedback from you, which helped us to evolve and shape our integration. Following your feedback, we are supplying you with all available Docker CLI tools to set up and mantain your continuous integration and continuous deployment workflows.

Getting Started

To start running your Docker project, all you need to do is add it on Semaphore.

After a brief analysis of your Git repository, Semaphore will pre-select the Docker-enabled platform, and you’ll be ready to use the docker CLI tools in your builds and deploys.

Other Use Cases for the Docker Platform

Since Semaphore’s Docker platform is based on a high-performance, fully virtualized environment, you can use it for tasks and tests that rely on low-level operating system operations, which are typically not possible in cloud-based CI, for example:

  • Developers at Project Calico are using Semaphore to simulate a full container networking environment by running Docker in Docker and multiple daemons;

  • The CoreOS team is using Semaphore to test their Rocket container project;

  • Developers working on systemd, a Linux init system used to bootstrap the user space and manage all processes, are relying Semaphore to run their tests.

Integrations with Docker Registries

Adopting Docker in continuous delivery adds some new steps in the process, one of them being the use of Docker registries to host produced container images as a base from which deployment to production takes place. If you are using Docker in production, chances are that you are using a container registry service to manage versions and host your container images.

Now you can use Semaphore’s high performance platform to build your containers faster than on any registry service. To make the CI setup process easy, we have also streamlined the authentication process required to access your private images in cloud-based or on-premise container registries. Check out the following documentation pages to learn how to set up authentication with a container registry:

We will be covering integrations with each of these registries in more detail in upcoming blog posts.

What Our Customers and Open Source Users Are Saying

Guilherme Íscaro, Software Engineer at Intel:

Soletta is an IoT framework that aims at multiple OSs and architectures. From time to time, some bugs were affecting only some architectures, and the worst part is that we only able to see the problem when someone was trying to use Soletta.

With the help of Semaphore and Docker, we were able to create Docker images for different architectures and run all our test cases. By doing so, we created a guarantee that Soletta was working on all architectures.”

Nick Ward, Director of Engineering at General Assembly:

General Assembly is a global education institution specializing in the most in-demand skills across data, design, tech and business. Everything that our engineering team touches can ultimately impact the experience of our global community - from students to instructors and from alumni to employers.

Semaphore’s Docker support allows our engineers at General Assembly to efficiently test and iterate over design choices with confidence. It grants us the ability to easily manage dependencies while decoupling our apps from the host environment.”

Tom Denham, Core Developer at Project Calico:

Calico provides networking for Docker containers. We’ve been looking for a cloud CI solution so that we don’t need to maintain the infrastructure for it. To run our tests, we need to simulate a full container networking environment by running Docker in Docker and multiple daemons. Semaphore is the only continuous integration service where we’ve been able to do that.”

David Jeusette, Senior Technical Architect, TextMaster:

TextMaster offers a professional online translation service that enables businesses to easily expand their international reach. More than 8000 companies in 110 countries trust TextMaster to optimise their translation workflow.

We run our dockerized applications on different cloud providers, and every time we commit and push new code to GitHub, we rely on Semaphore for CI. We have been using Semaphore to build images, render tests in the corresponding containers, pull images from private repositories, and apply blue-green deployment, as well as just to run tests in the dockerizing environment. We are really happy with Semaphore’s CI solution in every way.”

If you’re using or exploring Docker, we hope that you will find Semaphore’s Docker support useful in your work too. Take a look at our documentation for more info on our Docker platform and using Docker on Semaphore. If you have any feedback you’d like to share, we’d like to hear from you.

Happy building! 🐳

Platform Update on May 24th

The upcoming platform update is scheduled for May 24th, 2016.

Cassandra is updated to version 2.2.6.

Elixir receives an update with version 1.2.5.

Git is updated to version 2.8.2.

Go gets an update with 1.6.2.

Java receives an update with Oracle JDK version 8u91.

JRuby has been updated with version 9.1.0.0.

NodeJS receives four version updates with 5.10.1, 4.4.4, 0.12.14, and 0.10.45.

PHP gets three updates with 5.5.35, 5.6.21 and 7.0.6.

New things

The following additions can be used after switching to the release candidate platform.

Node.js 6.1.0 and 5.11.1 are now part of the platform. To use these versions, add nvm use 6.1 or nvm use 5.11 to your setup commands.

Ruby 2.3.1 and Ruby 2.2.5 have been added to the platform. These versions can be used by adding rbenv global 2.2 or rbenv global 2.3 to your setup commands.

Trying the new platform

To ensure that the updates are compatible with your current setup, please switch to the Ubuntu 14.04 LTS v1605 (release candidate) platform in Project Settings > Platform. We’re looking forward to hearing your feedback and requirements, which will help us to fix the potential issues and tailor the platform to better suit your needs. The release candidate period will last until May 24th, 2016.

Changes in the final release

The Docker-enabled platform gets several updates for its Docker tooling, namely: docker-engine 1.11.1, docker-machine 0.7.0, and docker-compose 1.7.1.

A full list of changes is available in the platform changelog.

Platform Update on April 26th

The upcoming platform update is scheduled for April 26th, 2016.

Elixir receives an update with version 1.2.4.

Erlang gets an update with version 18.3.

Git is updated to version 2.8.1.

Go gets two updates with 1.6.1 and 1.5.4.

Java receives an update with version 8u77.

JRuby gets two updates with 1.7.25 and 9.0.5.0.

MongoDB gets an update with version 2.6.12.

MySQL is updated to version 5.6.30 supporting the new --ssl-mode flag.

NodeJS receives three version updates with 4.4.3, 0.12.13 and 0.10.44.

PHP gets three updates with 5.5.34, 5.6.20 and 7.0.5.

RethinkDB has been updated to version 2.3.0, featuring the new users and permissions system.

New things

Node.js 5.10.1 is now part of the platform. To use this version during the release candidate period, add nvm use 5.10 to your setup commands.

Ruby 2.1.9 has been added to the platform. This version can be used during the release candidate period by adding rbenv global 2.1 to your setup commands.

Trying the new platform

To ensure that the updates are compatible with your current setup, please switch to the Ubuntu 14.04 LTS v1604 (release candidate) platform in Project Settings > Platform. We’re looking forward to hearing your feedback and requirements, which will help us to fix the potential issues and tailor the platform to better suit your needs. The release candidate period will last until April 26th, 2016.

A full list of changes is available in the platform changelog.

Protect Your Semaphore Account with Two-Step Verification

At Semaphore, we’re dedicated to protecting the security of our users’ data and building solutions to improve it. With that in mind, we’re very happy to announce that our newest security feature — two-step verification (also known as two-factor authentication) is now available on Semaphore.

Two-step verification secures your account by requiring a one time password, in addition to your username and password, to access your account. The goal of this extra step is to combine something you know (your password) with something only you would have access to (your phone). This ensures that your account stays secured even if your password is compromised.

Set up Two-Step Verification

You can enable two-step verification in your Semaphore account settings. Please refer to our documenation for a step-by-step guide to activating Semaphore’s two-step verification.

Once you have two-step verification enabled, you’ll need to enter a verification code, provided by your authentication application, to log into your Semaphore account.

One very important thing to note. As part of the set up process, you will be given a set of one-time recovery codes to use in the event that you are unable to access your phone.

You don’t have to use this method of securing your account, but it’s a good idea if you do. Stay safe!

Get future posts like this one in your inbox.

Follow us on