Semaphore Blog

News and updates from your friendly continuous integration and deployment service.

Testing Clojure With Expectations

Get future posts like this one in your inbox.

Follow us on

Clojure is shipped with clojure.test — a library with basic features for writing tests. However, there are a few alternatives that aim to make writing tests more pleasant or more suitable for BDD. Expectations by Jay Fields is one of them, described as “a minimalist’s unit testing framework” with the slogan “adding signal, removing noise”.

Expectations setup

The easiest way to get started is to add Expectations and lein-expectations to your project.clj:

(defproject expectations-playground "0.1.0-SNAPSHOT"
  :description "Playground for exploring Expectations"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [expectations "2.0.9"]]
  :plugins [[lein-expectations "0.0.8"]])

After that, you can run tests with lein expectations.

Expectations also integrates well with several editors and development environments, such as Emacs and IntelliJ. More information about installation and setup can be found in the library documentation.

Adding signal, removing noise

A simple example that compares clojure.test and Expectations already reveals few interesting details:

; clojure.test
(deftest equality-test
  (testing "Is 'foo' equal 'fooer'"
    (is (= "foo" "fooer"))))

; expectations
(expect "foo" "fooer")

Running the test with clojure.test gives:

While running the test with Expectations gives:

We already see Expectations delivering on the slogan “adding signal, removing noise”. The test is shorter, but it gives us more informations in a nicer way. Output is colored by default, format is a bit easier on eyes and it’s more precise about the failure — it says what’s different about strings. It also shows how much time the test took to run, which can be useful when optimizing tests.

Minimal and consistent syntax

It’s interesting to see how minimal and consistent the Expectations syntax is. At first glance, there is not much besides expect. It can get you a long way:

(expect 2 (+ 1 1))

(expect "foo" (str "f" "o" "o"))

(expect [1 2 3] (conj [1 2] 3))

(expect {:a 1 :b 2} (assoc {:a 1} :b 2))

(expect #"Expect" "Expectations")

(expect empty? [])

(expect 3 (in [1 2 3]))

Even more signal

Expectations really shines when it comes to testing collections as it tests not only equality, but also contents of collections:

Even less noise

Expectations tries aggressively to remove noise from test output. For example, it trims long stack traces, leaving only the important part.

Example stack trace using clojure.test:

This is the equivalent stack trace using Expectations:

Conclusion

Expectations certainly delivers on promise “more signal, less noise”. But the library also provides a few additional tricks that are not covered here — like testing side effects and freezing time. For more informations, visit the Expectations website.

Introducing Upcoming Platform Update, Selectable Platform Versions and Ubuntu 14.04 Support

The upcoming platform update is scheduled for Wednesday, September 24th, 2014 and is our first release to be based on Ubuntu 14.04 LTS.

With this update we’re introducing release candidate platforms which can be selected in project settings. From now on when we announce an update, you can immediately try it out.

Choosing between platforms in project settings.

We hope that this will ease the transition between updates. Keep in mind that these platforms are in a pre-release state and some things may not work as expected until the final release day. The current platform will be still available after the update so you can always fallback to it if needed.

We encourage you to send us your feedback and suggestions so that the final platform releases are in a sparkling condition and ready for you projects.

Platform version naming and update schedule

This month we’re beginning a more structured process of platform update management. Each version will be named according to a simple YYMM convention, for example this month’s version is 1409. We will release an update every month. If there are incremental or bugfix releases in the ongoing month, that version will have a minor increment, eg 1409.1.

Each month’s release is considered a major version. When a new major version is released, we will always keep the previous version available in project settings until the next major version release.

Projects using the latest major version will always be automatically upgraded to the new major version as soon at the time of its final release. Projects using an older major version will be upgraded to the latest one the moment their old platform version is dropped.

Ubuntu 12.04 end of life

To help all our users avoid any problems, this month we are releasing platform updates based on both Ubuntu 12.04 and 14.04. There will be no further platform updates based on Ubuntu 12.04; Semaphore platform version 1409 will be the last one. It will remain available until January 1st, 2015.

This month’s updates (v1409)

Firefox goes up to version 32, requiring bundle update selenium-webdriver for most projects that use that gem.

ElasticSearch is updated to version 1.3.2.

Ruby gets two updates with 1.9.2p330 and 1.9.3p547.

JRuby is also updated and now it’s running on version 1.7.15.

Bundler is now on version 1.7.2.

PHP updates include versions 5.3.29, 5.4.32, 5.5.16 and the addition of 5.6.0. For more information check out the changelog.

Git has been updated to version 2.1.

NodeJS receives two updates with 0.8.28 and 0.10.31

Updates for the final release

Ruby 2.1.3 has been added to the supported languages. Check out the Ruby changelog.

Bundler is now on version 1.7.3.

A full list of changes is available in the platform changelog

Semaphore stickers are here

At last they are here!

Semaphore continuous integration stickers

We have been very laborious and created Semaphore stickers in various sizes so they fit any occasion. After consulting with the stylists on the hottest trends for spring 2015 we present you our inspirational concept below.

To obtain stickers visit this page. We ship all around the world!

Continuos integration stickers for all occastions

Working With A Safety Net: A Developer Interview With Procore

In the “Developer Interview” series we talk to developers from some of the companies using Semaphore to find out how they work and share their insights with you. Here we had with us Michael Siegfried, Senior Engineer at Procore, one of the fastest growing companies in USA.

Tell us a little bit about your company and what it does.

Procore is a cloud-based construction software company. We aim to help construction companies work more quickly and efficiently. There are a lot of one-off tools for managing blueprints or contracts out there, but we give companies the ability to access all of their construction information in one place. A lot of our customers were keeping track of huge Excel files, so having all of their information available on any computer is huge for them!

“For any non-trivial feature we typically go through a research process that includes getting on calls with current customers”

What does your workflow typically look like, from an idea until it’s shipped?

For any non-trivial feature we typically go through a research process that includes getting on calls with current customers to suss out how it should behave. We have at least one member from the product, development, and quality assurance teams on these calls. That ensures that requirements don’t get passed down through hearsay and that we arrive at the best solution for all three departments.

If it’s a large project, we continue to work very closely with customers through a beta release to iteratively get closer and closer to the ideal feature. As development works on distinct parts of the feature, these branches will go through a quality assurance iteration before they are code reviewed and released.

What tools and guidelines do you use to write tests?

The two main resources for test-writing that we use are Thoughtbot’s guide to testing Rails applications and Better Specs. We write our specs in RSpec.

How do you approach refactoring, as the system grows?

One of the last refactorings I worked on was a large method called all_contract_buttons, which was, as the name implies, a method for gathering all of the buttons related to the contract. The method was 72 lines long and just had a bunch of duplication and irrelevant logic.

“Sometimes the act of refactoring costs more than the benefits of clean code.”

The way I approached the situation was to begin writing unit tests for the current implementation. Once I had this safety net, I began working on splitting up the logic and creating helper methods. The unit test became more of an integration test, but it enabled me to take this 72 line method and reduce it to 20 lines. The code was much easier to read and reason about. This approach gives you confidence that you are not breaking current functionality as you refactor.

What’s the last big realization about programming that you had?

One of the last big realizations about programming I had is that refactoring doesn’t necessarily justify its own cost. Sometimes the act of refactoring costs more than the benefits of clean code. It certainly makes sense to refactor something if you are touching that part of the code base, but sometimes there are bigger fish to fry.

How does Semaphore help you achieve your goals?

Semaphore is integral to our development process. We never deploy code to production unless the build is green in Semaphore. We also don’t merge branches into our master branch unless they are green.

A handful of times Semaphore has helped us find bugs in code that we were about to ship, but more often it helps us find bugs before we even consider the code ready to ship. Semaphore finds potential bugs before they become user-facing issues and gives us peace of mind that we are not breaking any tested code. While we have ways to go to get better spec coverage, it will only make Semaphore an even more important part of our process!

Ruby version usage in commercial projects, 2014 edition

Last year we calculated how much each version of Ruby was used by private projects on Semaphore. Given that many people viewed and shared the report, we decided to run the numbers again and compare the results.

To set the context, here are the most notable milestones achieved by the Ruby core team in the meantime:

  • Ruby 2.1.1 and 2.1.2 were released.
  • 1.9.3 entered security maintenance phase.
  • 1.8.7 and 1.9.2 reached end of life.

The chart below shows the distribution of Ruby versions in private projects tested on Semaphore.

Ruby versions used on Semaphore in 2014

The bar chart below shows difference in usage between 2013 and 2014.

Ruby versions used on Semaphore by years

The most notable change is that now the majority of projects is using some version of Ruby 2. If we combine the representation of all 2.x versions, we see a rise from 30.6% to 51.1%.

Semaphore Gets a Free Plan For Private and Open Source Projects

With great pleasure we are announcing that Semaphore is now free to use for private projects, for up to 100 builds per month. This is great for personal sideprojects, early stage MVPs and similar low volume development activities.

Free accounts have two processors available. This allows you to either test two private projects or branches at the same time, or run one build with tests split in two parallel threads. To ensure fair usage we are counting builds using two threads as two free builds.

We will convert all new users who are currently in a 30-day free trial to the new free plan.

Semaphore becomes free for open source

We are not stopping there however. The same free plan you can use for private projects also allows unlimited testing and deployment of open source projects.

Sign up, add your project from GitHub, run your first test suite in a minute without any changes in source code and share it on your profile.

Giving back to the community

Semaphore would not be possible without open source software. From the backbone essentials such as Git and Linux to tools like RSpec and PhantomJS, Semaphore is both built with and made to support development of projects based on open source.

Our goal has always been to make Semaphore free for open source projects. However bootstrapping from an almost anonymous position obliged us to wait until the business is profitable enough. All we had was a team and a goal to keep making the best continuous integration service we can. Now we are happy to share it with the community.

Language support

Semaphore currently has full support for testing and deploying Clojure, JavaScript / Node.js, PHP and Ruby projects. Our plan is to add support for all major open source languages by the end of the year.

Open source organizations

To allow large open source projects to take full advantage of our organization management features, we are introducing a special open source plan which allows an organization to have unlimited projects and unlimited open source builds with two processors for parallel testing.

Changes to existing profiles

All current users and organizations who already have an open source project on Semaphore will find that their project pages have become public. Only collaborators can access settings and trigger new builds and deploys, of course. For example, check out profiles of Perceptive Software or FlatStack.

Reduced price of Starter plan by $10, gets unlimited projects

To streamline our pricing and help you build more, we are making our Starter plan the first available paid plan. With unlimited private projects (instead of five) and one processor for running tests, it now costs $29.

We hope that you are as excited about the new capabilities as we are, and can’t wait to see what you’ll be making next. Happy building!

More Flexibile Connection Between Semaphore Projects And GitHub Repositories

We are glad to introduce a new feature within our project settings - “Repository” settings. The main purpose of this new settings option is to add more flexibility between your Semaphore project and GitHub repository.

Repository tab is placed within project settings. It allows you to see:

  • repository that this project is connected to
  • username of repository owner at GitHub
  • username of the Semaphore user whose token is being used for communication with GitHub
  • status of repository deploy key at GitHub
  • status of webhook that is set on Github.

Besides presenting you this information, the repository tab also lets you take action. For example, if a deploy key or webhook is missing on GitHub, Semaphore will present you an option to re-set them with a single click.

You can also change the repository used for the given project. This is useful if the repository name or owner on GitHub has changed and made the Git clone URL invalid. Or if the person whose OAuth API token Semaphore is currently using is no longer available. This way you will reactivate your project on Semaphore without losing project history, configuration and collaborators.

Introducing User Profiles

Recently we introduced organizations and organization profile pages with a list of projects. Today, we’re launching the first version of profile pages for users.

On the profile page you can see basic information about a user together with a list of her project. You can reach a user’s profile by clicking on her username in the project header on the dashboard. The profile is currently accessible only to the collaborators of one of the user’s projects.

This is only the beginning of the user experience overhaul that we started with organizations. We’re actively working on a new set of features so stay tuned.

Writing Clear Code: A Developer Interview With Agworld

Developer Interview” is a new series here at Semaphore blog. We’ll interview developers from some of the companies using Semaphore to find out how they work and share their insights with you. First up: Agworld, a company building software as a service for farm management. Chief scientist Jason Hutchens tells us their story.

Tell us a little bit about your company and what it does.

Agworld is an agriculture tech startup from Perth, Western Australia. We have customers around the world and offices in several other countries, including the US.

Our customers include both farmers and the agronomists, suppliers and contractors they work with. They use our website, iPad app and iPhone app to plan their season, manage their day-to-day operations, store historical farm records, share relevant information with stakeholders, produce detailed reports and view all of this data in the context of a map of their farm.

“Knowing that the work we do is moving us towards our goal of making global food production profitable and environmentally sustainable is a great motivator”

It may not sound sexy, but there’s a lot of really interesting problems to be solved in Ag, and knowing that the work we do is moving us towards our goal of making global food production profitable and environmentally sustainable is a great motivator.

What does your workflow typically look like, from an idea until it’s shipped?

Typically we speak to customers and potential customers directly to understand the problems that we could help them to solve, even sending developers on field trips to experience all of that first-hand. This allows our product managers to work closely with our development and customer support teams to sketch out a solution, working with our designers to mock UI and often writing user stories in the form of cucumber scenarios.

We embrace Scrum for development, and integrate weekly for internal testing. We release frequently, with each release going through manual QA before being pushed to production in each of our geographical regions.

We maintain separate feature and hotfix branches across dozens of repositories, as our product is heavily service-oriented. Merging is typically done via a pull request on GitHub, giving us peer oversight, and Semaphore integration helps us to perform these merges once the build is green.

What tools and guidelines do you use to write tests?

In general we endeavour to make sure that all new code has sufficient test coverage; this is usually achieved through peer review rather than a formal process, although we’ve recently taken advantage of the Coveralls integration with Semaphore to easily eyeball our code coverage. This has also helped us to remove stale code (our app dates back to 2009, so we refactor as we go to reduce technical debt).

We use the fairly standard combination of RSpec of unit tests and Cucumber, Capybara, PhantomJS for integration tests.

“Your code should reflect the fact that it’s solving some real-world problem for a real-world customer”

What’s the last big realization about programming that you had?

Personally I’ve had two big insights recently:

You should write code as if you’re explaining what you’re doing to another developer. Your goal should be to eliminate developer uncertainty about what the code is doing. Sections of your codebase that are especially defensive (evidenced by lots of conditional code that checks the state of variables) belies this lack of trust, and should be fixed.

You shouldn’t write any code until you properly understand the problem that you’re solving. It’s easy to get distracted by a great solution without respecting the underlying problem. Your code should reflect the fact that it’s solving some real-world problem for a real-world customer. Business logic should be clearly evident; you shouldn’t have to go looking elsewhere to find out why the code exists.

When was the last time Semaphore saved you?

Semaphore saves us in little ways every day; we don’t need to maintain our own in-house build farm, the status of our branches is always visible, and assembling a release candidate from a dozen feature branches is no longer an intractable problem. Also, it’s great that setting up builds for a wide variety of projects across JRuby, MRI 1.9.3 and 2.1, and JavaScript required no management and very little manual configuration.

Introducing Organizations

Today we are very happy to announce Organizations on Semaphore. Organizations simplify project management inside companies and other groups, introduce a fine grained permission system and help you focus on your work while Semaphore scales up the continuous integration process as your operation grows.

If you ever wanted to let somebody else add a project under your billing account or give a group of collaborators read-only access without ability to launch builds and deploy, you’re going to love Organizations.

We’re also introducing some new payment options that are tailored for companies and enterprises, so let’s dive in what’s new.

One account for your organization’s CI needs

Creating an Organization lets you group all projects that belong to your organization under a new billing account. You create an Organization as a regular Semaphore user and become its owner, with ability to invite other users to share the ownership. Being an owner gives you full access to all projects and all organization management features, from billing to permission settings.

Adding a project to an organization

Once you have or belong to an organization, you can choose to add it to the organization account.

You can easily transfer an existing project under your personal account to an organization. To do that, go to your Account, locate your project and select the organization to transfer it to:

All existing collaborators will be copied to a new organization team with access to the given project.

Projects are accessed by teams with permissions

As an organization owner, you can define teams of users which have access to selected projects.

You can invite new members to a team by email. Similarly to the new collaboration model we launched recently for personal projects, team access is not connected to who is a collaborator on GitHub, giving you flexibility to add or remove anyone. This is also one of the reasons we are now requiring all users to have a Semaphore username and using the username as the identity indicator once a user is on board.

Teams can have one of the following permission sets:

  • Access — Team is only able to access projects and read build and deploy results.
  • Edit — In addition to Access level, team can also change project settings, add branches and manually trigger builds and deploys.
  • Admin — In addition to Edit level, team can also add and remove projects from the organization.

You can of course have as many teams as you want.

Besides the interface to manage teams, you can also see and manage individual organization members.

Profiles

Organization profiles are our first step towards making Semaphore more user-centric. Since project’s owner is part of its identity, you can now click on the organization name and view its most recently updated projects.

Pricing

With Organizations we’ve updated our pricing structure.

Personal plans include Solo, Starter and Double, formerly known as Pro 2, now with unlimited projects.

If you’re currently using one of the plans with more processors, you can continue to do so indefinitely. Previous higher plans have their equivalent in organization plans, at the same price, since we believe you will benefit from using Organizations.

Organization plans start with Pro 2 and go up to Pro 32. They have no project limit, and the only variable is the number of processors for concurrent and parallel builds, with higher plans including a discount instead of having a price that is a simple multiplier.

We are introducing an option to pay annually on all plans. This is great if you want to simplify the charge flow and enjoy a discount equivalent to getting one month of free service.

Organizations can also pay annually by invoice. To do this, simply go to your organization’s subscription settings and submit the company information form. We will invoice you on net 30 terms with wire transfer instructions.

Take a look at our new pricing.

Get started with Organizations today

It took us a few internal iterations to get to this solution which we believe will scale for organizations of all size and the time to come. We are happy to grow with you, dear user, and hope that these features will help you and your team deliver some amazing software.

Create a new organization now.

Get future posts like this one in your inbox.

Follow us on