NEW Run your Ruby tests in a few minutes and ship faster with Boosters, one-click auto parallelization · Learn more…
×

Semaphore Blog

News and updates from your friendly continuous integration and deployment service.

New Server Page: A Better Overview of Continuous Deployment

Get future posts like this one in your inbox.

Follow us on

Setting up deployment on Semaphore allows your team to have a uniform and easy to use software delivery pipeline. To that end, Semaphore provides you with a concept of servers. These are essentially configured deploy targets which let you share history of who deployed what and when. Today we’re happy to present you a brand new look that we believe will provide an improved experience.

So far servers on Semaphore reflected the pipeline that they’re part of, which begins with a Git commit, goes through a CI build on Semaphore and finally ends with a deploy, triggered manually or automatically. Looking at that fixed three-step pipeline over time, we’ve realized that it contains a lot of machine-level information that’s pushing out what’s most interesting about it: the actual change your fellow developer shipped. And that’s exactly what we focused on in the new design.

Continuous deployment overview on Semaphore CI

The new activity stream more prominently emphasizes the deploy message and its author. Since most commits that trigger deployment are a result of pull request merges on GitHub, Semaphore is smart to recognize those and display the human-entered message that is originally bellow the automatically generated merge commit summary.

Since a team can use a server page as the one true changelog, Semaphore makes it easy to customize the message of each finished deploy. To do that, just open any individual deploy and click on its title to edit the title and description.

Editing a deploy message on Semaphore CI

In implementing the new design we’ve also made sure that it is responsive and looks great on your phone as well as on your computer.

If you haven’t set up deployment on Semaphore yet, it is very easy to do. Here’s a guide to help you get started:

We’ll be rolling out the new design to all users over the next couple of days. We’re super happy to share it with you and can’t wait to keep iterating on it. Let us know how you like it in the comments or by contacting us on support. Enjoy, and happy building!

New Dashboard: Simpler, Faster, Responsive

The Semaphore dashboard is about to get a brand new look. The new design dramatically simplifies the layout, while improving performance under the hood. Over the next couple of days, we’ll be rolling out this update to all users.

Previously, the dashboard included all projects that you had access to, including the status across all servers and recent branches. Their order would change based on last activity — the ones with new builds or deploys would move to the top. Even though we’ve introduced the option to collapse projects, after you’ve used Semaphore for a while, this behavior often becomes suboptimal. There’s just too much information on the screen, with its order out of your control.

How it works

The new dashboard focuses on two things:

  1. Communicating the status of each project’s main branch, and
  2. Providing an easy way to reach the projects you’re interested in.

Continuous deployment dashboard on Semaphore

Each project forms one uniform row, with an indicator of build status for its default branch. The default branch is either master or the first branch you’ve added to Semaphore. You can change a project’s default branch in “Project Settings > Branches”.

You can open the latest build by clicking on the status icon next to the project name, or you can click anywhere within the row to open the project.

By default, projects are sorted alphabetically. However, you can mark a project as a favorite by clicking on the star on the right-hand side. Favorite projects will always be displayed first, but after you’ve marked a few, you may want to toggle the view to display only your favorite projects. You can do that by clicking on the star button at the top of the project list. Semaphore will remember your preferences the next time you visit the dashboard.

Continuous delivery dashboard on Semaphore, showing only favorites

As a bonus, the new dashboard is fully responsive, making it handy to view on your phone.

We’re very excited to share the new design with you. Let us know how you like it in the comments or by contacting us on support. Enjoy, and happy building!

Platform Update on January 19th

The upcoming platform update is scheduled for January 19th, 2016.

Erlang has been updated to version 18.2.1.

Git has been updated to version 2.7.0.

Heroku toolbelt is now on version 3.42.27.

Node.js gets an update with version 4.2.4.

PHP receives two updates with versions 5.6.17 and 5.5.31.

Redis gets an update with version 3.0.6.

RethinkDB is now on version 2.2.3.

This update also includes optimized Ruby 2.2.x versions, which should yield noticeably improved build times for projects configured to use one of these versions.

New things

Elixir 1.2.0 has been added to the platform. It incldes a wide range of performance improvements, bugfixes and other features. This version can be used during the release candidate period by adding kiex use 1.2.0 to your setup commands in Project Settings.

Node.js 5.4.0 is now also part of the platform. To use it during the release candidate period, add nvm use 5.4 to your setup commands in Project Settings.

Ruby 2.3.0 has been added to the platform. This long awaited Ruby version packs quite a few improvements and additions like the safe navigation operator and the frozen literal pragma. Ruby 2.3.0 can be used during the release candidate period by adding rbenv global 2.3 to your project’s setup commands.

Trying the new platform

To ensure that the updates are compatible with your current setup, please switch to the Ubuntu 14.04 LTS v1601 (release candidate) platform in Project Settings > Platform. We’re looking forward to hearing your feedback and requirements, which will help us to fix the potential issues and tailor the platform to better suit your needs. The release candidate period will last until January 19th, 2016.

Changes in the final release

Go gets an update with version 1.5.3.

A full list of changes is available in the platform changelog.

Fast Continuous Delivery of Microservices with AWS Lambda

AWS Lambda is an AWS service that runs your code in response to events or HTTP requests. Lambda works through functions, which are basically microservices written in Java, Python or Node.js. When you create a Lambda function and deploy your code to it, AWS Lambda takes care of provisioning and managing the backend infrastructure.

Continuous deployment to AWS Lambda with Semaphore

To make it easy to get new versions of your code running on AWS Lambda as quickly as possible, today we’re happy to announce an integration on Semaphore for AWS Lambda users. Here’s a brief screencast that shows how to set up continuous deployment to AWS Lambda in a few easy steps.


Our goal has been to simplify the process as much as possible. With that in mind, Semaphore can detect your functions on AWS Lambda and guide you through the rest of the process with little input required.

You can tell Semaphore to automatically deploy your application from a selected branch whenever all tests pass, or you can choose to do it manually. Your team will have a shared deployment history available at any time and an easy and uniform way to deliver new versions of your microservices.

To help you get started from scratch, we have also published a detailed tutorial on how to build, test and deploy a microservice which sends an SMS message to those who break a build on Semaphore using AWS Lambda:

We hope this integration will help you speed up your continuous delivery workflow.

In case you missed it, we have recently also added deployment integrations for Amazon S3 and AWS Elastic Beanstalk.

Happy building!

Node.js Version Usage in Commercial Projects, 2015 Edition

We’ve been publishing Ruby version reports in commercial projects for three years now, and given recent events in Node community — the merge of io.js with Node.js, appearance of v4.0.0, faster release cycles under a new development policy — we thought it’d be interesting to see what exactly people use to run their production apps today. So here are the stats.

Node.js version usage for commercial projects on Semaphore in 2015

The majority of private projects (73%) are using Node 0.x. Of course, we’re committed to making the latest versions available to all our users through our monthly platform updates.

What’s your team’s approach to keeping up with new Node.js releases? Feel free to discuss in the comments below.

Platform Update on December 22nd

The upcoming platform update is scheduled for December 22nd, 2015.

Cassandra gets and update with version 2.2.4.

Git receives an update with version 2.6.4.

Go gets an update with version 1.5.2.

Heroku toolbelt receives an update with version 3.42.25.

Java gets one update with version 7u91.

Maven is getting updated to version 3.3.9.

MySQL gets an update with version 5.6.28.

Node.js gets three important security updates with 0.10.41, 0.12.9 and 4.2.3. These updates include the latest security fixes for OpenSSL.

PHP receives an update with version 5.6.16.

New things

Two new Node.js versions have been added to the platform: 5.1.1 and 5.2.0. To use these in your builds during the release candidate period, add nvm use <version> as a setup command in Project Settings, where the value for <version> is either 5.1.1 or 5.2.0. On the day of the final release of the new platform, these versions will be selectable in Project Settings too.

PHP 7.0.0 is now part of the platform. Add the following setup command to your project to use it: phpbrew use 7.0. This version will be selectable in Project Settings too, after the release candidate period.

Trying the new platform

To ensure that the updates are compatible with your current setup, please switch to the Ubuntu 14.04 LTS v1512 (release candidate) platform in Project Settings > Platform. We’re looking forward to hearing your feedback and requirements, which will help us to fix the potential issues and tailor the platform to better suit your needs. The release candidate period will last until December 22nd, 2015.

Changes in the final release

Node.js receives another addition with version 5.3.0.

Ruby gets two additions with versions 2.1.8 and 2.2.4.

A full list of changes is available in the platform changelog.

Ruby Version Usage in Commercial Projects, 2015 Edition

Continuing our modest tradition of publishing an annual report on Ruby versions used for private projects on Semaphore, we’re presenting you with the results for 2015.

Ruby version usage for commercial projects on Semaphore

Since we published last year’s report, we’ve seen a steady flow of new Ruby releases according to the decision to adopt the semantic versioning policy.

Given the multitude of Rubies 2.1.x and 2.2.x now available, we’ve decided to present them grouped as such in the overview chart above.

It appears that once a team has managed to cross the chasm between 1.9.x and Ruby 2, adopting newer versions is relatively easy. Today, 79% of all commercial Ruby projects are using some flavor of Ruby 2, up from 51% last year. The following chart shows the adoption cycles for each version:

Ruby version adoption year over year on Semaphore 2015

Of course, life is complicated, and old versions don’t just go away. A closer look shows increasing fragmentation in Ruby versions used among developers:

Ruby version fragmentation in commercial projects on Semaphore 2015

As for Rubies 2.1.x and 2.2.x, even though it is now much easier to upgrade to a newer micro version, zooming in the data shows that usage is quite fragmented too:

Ruby 2.1.x and 2.2.x usage on Semaphore 2015

What’s your team’s approach to keeping up with new Ruby releases? Feel free to discuss in the comments below.

Introducing Semaphore Insights: Find Test Files That Slow You Down

Today we’re happy to announce Semaphore Insights, a new part of Semaphore with the goal to give you useful, actionable feedback on your continuous integration process.

Semaphore Insights: Find slow test files

As developers we all want our builds to finish fast, but sometimes we introduce bottlenecks to the test suite without realizing it. This can be by writing tests that make excessive database calls or similar inefficient use of resources. Just like we understand the need for refactoring the application code, we need to find time to refactor and improve our test code too.

So the first available feature in Semaphore Insights is a report on your slowest test files. It displays all test files which take more than a minute to run, and if none such are found then the slowest five by default.

Semaphore Insights: Example list and continuous integration history chart for Cucumber test files

We have found that a minute is a sensible threshold in large projects — more than that is usually a smell and we recommend looking into the code or considering to split the test file. The benefits are both a faster feedback loop your development workflow and parallel execution in CI. For example, if you have a test file that takes 4 minutes to complete, then your whole build cannot complete sooner than 4 minutes, plus some setup and teardown operations that may exist for your project.

How to enable Semaphore Insights

On your project page, click on “Find slowest test files” link and then enable the feature for your test framework. That’s it — the report will be ready after your next successful build. To see the chart of historical performance, just click on a test file to uncover it.

In order to gather the necessary information, Semaphore inserts some additional formatting options to the corresponding test framework’s configuration file, such as .rspec. That should not affect your build in any way, but if you think it does, report it through a support message and we’ll investigate.

We will be rolling out this feature to all users over the next couple of days.

Semaphore Insights currently supports RSpec 3 and Cucumber. If your testing framework is not supported but you would like it to be, we want to hear from you.

We hope that you will find Semaphore Insights useful. If you have any feedback on it, or even have ideas where it could go next, please get in touch.

Happy building!

Platform update on November 24th

The upcoming platform update is scheduled for November 24th, 2015.

Chromedriver gets an update with version 2.20 which supports Chrome versions v43-48.

Git has been updated to version 2.6.3.

Java gets two version updates with 7u85 and 8u66.

PHP receives an update with version 5.6.15.

RethinkDB gets an update with version 2.2.0.

RubyGems has been updated to version 2.5.0 for all Ruby versions supported by Semaphore.

New things

Elixir 1.1.1 has been added to the platform. Until this version isn’t added to the language settings for your project, adding kiex use 1.1.1 as a setup command will activate it for your builds.

Firefox 38.4.0esr has been added along with a tool for switching between different versions. To change the currently active Firefox version, add change-firefox-version <version> as a setup command. Valid versions are 38 or esr for Firefox 38.4.0, and 34 or default for Firefox 34.0. The default version is 34.0.

Node.js 5.0.0 is now part of the platform. To use it in your builds during the release candidate period, add nvm use 5.0 as a setup command in Project Settings. On the day of the final release of the new platform, Node.js 5.0.0 will be selectable in Project Settings too.

Python 3.5 has been added to the platform.

Trying the new platform

You can evaluate the new platform right away by selecting Ubuntu 14.04 LTS v1511 (release candidate) from the Platform section of Project Settings. We encourage you to try it out, so that we can fix the issues you may encounter until the final release on the next Tuesday.

Changes in the final release

RethinkDB gets an update with version 2.2.1.

RubyGems 2.5.0 has been rolled back to version 2.4.8 due to incompatibilities with some gems.

A full list of changes is available in the platform changelog.

Bootstrapping a New Product a Developer Interview with Rupeal

In the “Developer Interview” series we talk to developers from some of the companies using Semaphore to find out how they work, and share their insights with you. This time we had the pleasure of talking with Pedro Pereira Santos, team leader at Rupeal, a Portuguese company that has two completely bootstraped products behind them.

Tell us about your role and responsibilities at RUPEAL.

Pedro Periera Santos ClanHR

I’ve been a team leader for a couple of years. I work together with the team on building new features, planning our product’s architecture and improving our development process. When I moved to the company, I started doing Ruby on Rails on InvoiceXpress, an online invoicing system. Three years later, I am now bootstrapping again with a new product: ClanHR.


What does your product design and development workflow for ClanHR look like? Any important lessons from the past you’re applying to your new product?

We have a product owner that does market research and builds proper specs. We use marvelapp to build interactive mockups, and we show them to clients for product fit. Next, our designer builds mockups of the features and iterates on them until we are happy with it. The designer is proficient in building ReactJS/HTML/SasS applications, and she can build a visual representation of the features in ReactJS. This means that we should have proper specs and already available pages and React components when the features get to the developers. The developers just need to add functionality.

ClanHR Dashboard

All code is on GitHub, and all changes must be made via pull requests. When a pull request is available, the entire team must review it and OK it. As soon as the branch is created, Semaphore steps into action: it bootstraps the app, checks dependencies, runs the tests, and generates a package. If everything is working, the pull request will be marked as good to merge.

When we merge the pull request, Semaphore automatically deploys the master version to the staging environment. This means that we get to staging very quickly. This is important because the product owner and the designer can fine-tune the features quickly, and it saves us time spent on deploying.

This is in contrast with our other product. Due to several legacy issues we can’t be this agile, and end up spending a lot of time waiting around for things to happen.

Is your team doing TDD or BDD? How would you describe your role of Lead Code Detonator in the process of product development?

We mostly do TDD. When the developers get a feature they need to implement, we usually team up to think things through together. Even if it’s just going to be one developer to implement the feature, this analysis phase really boosts our productivity. Next, the developer starts coding. We are working with Clojure, and have a very fast workflow with fast unit tests. We implement endpoints on the services without booting a server, we just create tests and scenarios until we have the server side of the feature done.

The last step is to go to the frontend and connect everything.

ClanHR Team Photo

What tools and guidelines do you use to write tests across your application stack?

Because we rely so much on TDD, our tests’ feedback cycle needs to be pretty fast. When we are testing code with dependencies like the database, we create a gateway for the actual database, and an in-memory implementation that mimics the database gateway. We then implement and test this very fast environment again. However, we also want to test all that we are doing against the database gateway. We can do that on our development machines, but usually we leave that to Semaphore. When we push, Semaphore runs the tests two times, one for each gateway.

We also try to focus on gateway-less scenarios. Using functional programming and focusing on pure functions allows us to build tests that just check data and have no dependencies.

All services are tested as a unit, and they don’t communicate with other services during the test process. However, we use Nightwatch to test features on the frontend. These are very high-level tests that touch all parts of the application: the frontend, services and the database. They are very slow, so we try to keep them small and only create scenarios for key workflows in the application.

You mentioned to us that your full tests went down from 40 minutes on our machines to 15 minutes on Semaphore. How did this happen?

At one point, our full test suite for InvoiceXpress would take about 40 minutes to complete. This was on our development machines, and we started running the full suite only before merging a branch. However, on Semaphore, the full build took 30 minutes to do the same! And then we enabled parallel execution and split the unit tests from the Cucumber tests, and the full suite took 15 minutes.

“The feedback cycle improved so much that we started pushing to master to verify that the tests were passing.”

ClanHR Team Photo

How does Semaphore help you achieve your goals?

Semaphore is very easy to configure, and also very powerful. It enables us to easily implement a continuous delivery process without having to worry much about the way to implement it. Any team member, even the non-programmers, can make changes, and as soon as the branch is merged, it will take only a couple of minutes for them to see their changes live.

The support is also great and fast, and always tries to come up with solutions for our weird ideas. The whole company feels that Semaphore is a valuable business partner.

Get future posts like this one in your inbox.

Follow us on