NEW Run your Ruby tests in a few minutes and ship faster with Boosters, one-click auto parallelization · Learn more…
×

Semaphore Blog

News and updates from your friendly continuous integration and deployment service.

Setting up the BDD stack on a new Rails 4 application

Get future posts like this one in your inbox.

Follow us on

Note: An updated version of the tutorial is available at Semaphore Community.

Semaphore has been, and continues to be, developed using the behavior-driven development process. In this post, I will take you through the steps needed to set up our prefered BDD stack on a new Rails 4 application and explain why each tool is important.

Create the application

$ rails new myapp

Install RSpec

We prefer RSpec to other testing frameworks since we find it more natural and expressive. For example, I never find myself using the word “assert” in everyday talk, while I use the word “should” regularly. Also, RSpec’s language for writing tests and output is very readable and can serve as documentation.

Add rspec-rails gem to the development and test groups of your Gemfile.

group :development, :test do
  gem 'rspec-rails', '~> 2.0'
end

Install the gem:

$ bundle install

Bootstrap the app with RSpec:

$ rails generate rspec:install

Create the RSpec binstub. In short, the binstub will allow you to run RSpec with bin/rspec instead of bundle exec rspec:

$ bundle binstubs rspec-core

Install shoulda-matchers

shoulda-matchers lets us spec common Rails functionality, like validations and associations, with less code.

Add shoulda-matchers gem to the test group of your Gemfile:

group :test do
  gem 'shoulda-matchers'
end

Install the gem:

$ bundle install

Let’s take a look at a simple validation spec.

If you are validating the presence of post’s title:

class Post < ActiveRecord::Base
  validates :title, presence: true
end

without shoulda-matchers the spec might look something like the following:

require 'spec_helper'

describe Post do

  describe "title validation" do

    context "title is present" do

      before(:each) do
        @post = Post.new(title: "My first post")
      end

      it "does not add an error on the 'title' attribute" do
        @post.should have(0).error_on(:title)
      end

    end

    context "title is not present" do

      before(:each) do
        @post = Post.new
      end

      it "adds an error on the 'title' attribute" do
        @post.should have(1).error_on(:title)
      end

    end

  end

end

and with shoulda-matchers:

require 'spec_helper'

describe Post do
  it { should validate_presence_of(:title) }
end

Install Factory Girl

Factory Girl is “a library for setting up Ruby objects as test data” or more precisely it is a fixtures replacement.

Add factory_girl_rails gem to the development and test groups of your Gemfile:

group :development, :test do
  gem 'rspec-rails', '~> 2.0'
  gem 'factory_girl_rails'
end

Install the gem:

$ bundle install

Basically, Factory Girl will allow you to create objects that you need in your tests without providing a value for each required attribute. If you don’t provide a value for a required attribute Factory Girl will use a default value that you defined in factory’s definition.

Factory Girl also has a more pleasant system for defining record associations than when using fixtures.

Let’s define a post factory:

FactoryGirl.define do
  factory :post do
    title "My first post"
    content "Hello, behavior-driven development world!"
  end
end

Now, if both the title and content attributes are required to create a valid post, instead of writing in our spec something like the following:

require 'spec_helper'

describe Post do

  describe "creation" do

    context "valid attributes" do

      it "should be valid" do
        post = Post.new(title: "My first post", content: "Hello, behavior-driven development world!")

        post.should be_valid
      end

    end

    context "invalid attributes" do

      it "should not be valid" do
        post = Post.new(title: "My first post", content: "")

        post.should_not be_valid
      end

    end

  end

end

you can just write:

require 'spec_helper'

describe Post do

  describe "creation" do

    context "valid attributes" do

      it "should be valid" do
        post = FactoryGirl.build(:post)

        post.should be_valid
      end

    end

    context "invalid attributes" do

      it "should not be valid" do
        post = FactoryGirl.build(:post, title: "")

        post.should_not be_valid
      end

    end

  end

end

Make sure everything is connected and working

Create a Post model:

$ rails generate model Post title:string content:text

      invoke  active_record
      create    db/migrate/20130726125040_create_posts.rb
      create    app/models/post.rb
      invoke    rspec
      create      spec/models/post_spec.rb
      invoke      factory_girl
      create        spec/factories/posts.rb

Notice, the generator now also creates a model spec and a ‘posts’ factory. That’s the reason why we included the rspec-rails and factory_girl_rails gems in the development group of the Gemfile.

Update the spec to validate post’s title and content:

require 'spec_helper'

describe Post do

  it { should validate_presence_of(:title) }
  it { should ensure_length_of(:title).is_at_least(5) }
  it { should validate_presence_of(:content) }
  it { should ensure_length_of(:content).is_at_least(10) }

end

And update the Post model with validation definitions:

class Post < ActiveRecord::Base

  validates :title, presence: true, length: { minimum: 5  }
  validates :content, presence: true, length: { minimum: 10  }

end

Before running the spec make sure to apply the migration and prepare the test database by recreating it from db/schema.rb.

$ bundle exec rake db:migrate db:test:prepare

After running the spec you can see it pass:

$ bin/rspec spec/models/post_spec.rb

Install Cucumber

Cucumber helps us both focus on the feature-level and as a high-level integration testing tool.

Add cucumber-rails gem to the test group of the Gemfile.

group :test do
  gem 'shoulda-matchers'
  gem 'cucumber-rails', require: false
  gem 'database_cleaner'
end

You can also add the database_cleaner gem which is not required, but it will save you a lot of heartache. It’s used to ensure a clean database state for testing.

Install the gems:

$ bundle install

Bootstrap the app with Cucumber:

$ rails generate cucumber:install

Create the Cucumber binstub:

$ bundle binstubs cucumber

Install selenium-webdriver

To be able to run Cucumber scenarios which use Javascript you need selenium-webdriver.

Add it to the test group of your Gemfile:

group :test do
  gem 'cucumber-rails', require: false
  gem 'database_cleaner'
  gem 'factory_girl_rails'
  gem 'selenium-webdriver'
end

And install it:

$ bundle install

Make sure Cucumber is working correctly

To do that, let’s develop a simple feature.

# features/home_page.feature
Feature: Home page

  Scenario: Viewing application's home page
    Given there's a post titled "My first" with "Hello, BDD world!" content
    When I am on the homepage
    Then I should see the "My first" post
# features/step_definitions/home_page_steps.rb
Given(/^there's a post titled "(.*?)" with "(.*?)" content$/) do |title, content|
  @post = FactoryGirl.create(:post, title: title, content: content)
end

When(/^I am on the homepage$/) do
  visit root_path
end

Then(/^I should see the "(.*?)" post$/) do |title|
  @post = Post.find_by_title(title)

  page.should have_content(@post.title)
  page.should have_content(@post.content)
end
# config/routes.rb
Myapp::Application.routes.draw do

  root to: "posts#index"

end
# app/controllers/posts_controller.rb
class PostsController < ApplicationController

  def index
    @posts = Post.all
  end

end
<!-- app/views/posts/index.html.erb -->
<ul>
  <% @posts.each do |post| %>
    <li>
      <%= post.title %><br />
      <%= post.content %>
    </li>
  <% end %>
<ul>

Now run the feature file and you should see it pass:

$ bin/cucumber features/home_page.feature

Congratulations for making it this far. You should now be fully equipped to work in the BDD cycle and deliver clean, working code.

Introducing continuous deployment

Semaphore is great for testing your web application. Every day, code of thousands of developers is flowing through Semaphore for verification so that they can deliver new features to users with confidence.

But there’s more to delivery than testing, namely you need to deploy your code to the target servers. If you’re working in a team, someone needs to make sure that everyone has the right SSH configuration on their machines to be able to deploy. Done manually, deployment is a repetitive and distracting procedure. And since it has to be done within terminal sessions, there’s no way of keeping track of who deployed what and when.

Today we’re officially launching deployment as a first-class feature to futher increase your development velocity. Now you can let Semaphore automatically deploy working code to your application server. You’ll know exactly which version is on each of your servers. Best of all, this comes completely free - deployment jobs have top priority and do not count against your processor limit.

Here’s how it works:

Setting up a server

Get started by going to your project settings, Deployment tab. Following our goal of making things as simple as possible, you can set up a deploy target in a few easy steps. We are launching with two major options: Heroku and a custom script, which is usually Capistrano.

When using your deploy script, you need one new passwordless SSH RSA key that is present in the .ssh/authorized_keys file on server, inside the deploy user’s home directory.

Is it automatic?

The major behaviour difference comes from this option. Automatic deploys are tied to a particular branch, so for example every passed build of the staging branch will be deployed to this server.

With manual deployment, you can deploy any build from any branch. Note that this action is still possible to do on servers with automatic deployment. For example, you normally deploy to staging server from the staging branch, but sometimes also a snapshot of a short-lived feature branch.

Green builds trigger an automatic deploy

With automatic deployment set up for a specific branch, Semaphore will run a deploy with commands that you’ve provided in the setup process. The deploy in progress and report look similar to the build counterpart.

Deploying manually

To deploy a particular build manually, open it and click on “Deploy manually” in the top right corner of its header. You can then select the target server and write a quick release note. This is very useful because in many cases the last commit message will be something like “Merge ‘staging’ into master”. Instead of that, you can write something which will make it clear what went in with the release.

Notifications come in full package

The project timeline now includes deploys and their status for your viewing pleasure.

Semaphore can send email and chat notifications for each deploy. As with builds, you can turn this off in your project’s notification settings.

For a long time we’ve been sending reports to the login email address, but now every user can specify a different email for notifications on each project.

Deploys also come with full API and webhooks support.

A sidenote for the API: we’re aware that it could be a little bit better - we’ve started with a single endpoint and deliberately been adding new layers over time - so a V2 is in plans.

Viewing deploy history

While the dashboard and project page display the current deploy on each server, when necessary you can go to the server page and see the full deploy history. The entries are separated by dates which is a nice way to get a sense of progress.

We hope that you will enjoy these new features. Our big thanks goes to our power users who have been deploying their apps while this was in beta.

Ship early and ship twice as often.

Exploring CasperJS and Continuous Integration

All Ruby developers are probably familiar with Capybara - the tool for interacting with web pages that we all use and love. A similar thing exists in the JavaScript world - CasperJS.

CasperJS is a navigation scripting and testing utility for PhantomJS.

Since the setup is really simple, I wondered if CasperJS can be used for testing a Rails application and more so, how hard it is to run those tests on Semaphore.

Simple example

Here is a single test that visits the posts page and checks if everything is in order:

// test/javascripts/posts.js
casper.test.begin('Posts', 2, function suite(test) {
  casper.start("http://0.0.0.0:3000/posts", function() {
    test.assertTextExists("Posts");
    test.assertTextExists("Here you can find all posts");
  });

  casper.run(function() {
    test.done();
  });
});

The full example repository can be found here.

Semaphore setup

The setup includes a few steps:

  • preparing the Rails application
  • installing CasperJS (PhantomJS is already available on Semaphore)
  • loading test data
  • running the Rails application server
  • executing CasperJS tests

This translates nicely into several Semaphore build commands:

bundle install --deployment --path vendor/bundle
bundle exec rake db:setup
bundle exec rake db:test:prepare
git clone git://github.com/n1k0/casperjs.git ~/casperjs
cd ~/casperjs
sudo ln -sf `pwd`/bin/casperjs /usr/local/bin/casperjs
cd -
bundle exec rails server -p 3000 -e test &
sleep 30 # wait for server to run
casperjs --direct test test/javascripts/posts.js

Result

The result looks pretty green:

It’s worth noting that in this setup CasperJS tests don’t run in isolation: there are no database transactions so state persists across the whole test suite.

The right tool for the job

Is CasperJS a good fit for testing Rails applications on Semaphore? Probably not… There are other tools that can serve that purpose. But if you are looking for a tool for smoke testing a live instance of your app or you are more into JavaScript than Ruby, CasperJS is worth looking into.

If you do want use CasperJS in a Rails project, consider crafting a Rake task that can serve a similar purpose as the build commands from above.

What cool and unusual tools are you using in your project? Anything interesting in your Semaphore build commands? Let us know!

Book review: Growing object-oriented software, guided by tests

growing object-oriented software book cover

Growing object-oriented software, guided by tests was published in 2010 and is already considered a classic in the TDD community. I have found this to be for a good reason. It is the first book that I am aware of that guides you through development of a non-trivial application using TDD as a paradigm.

It begins with a short introduction of testing practices, where topics such as the TDD cycle and “tell, don’t ask” principle are briefly discussed. These are now well known in the testing world and are not covered in-depth; for those who seek more information there are references to explore further. Testing is just a part of the system design process, so there are some guidelines for object-oriented design as well.

The authors, Steve Freeman and Nate Pryce, then quickly dive in developing a working example: a Java desktop application that watches online auctions and bids over XMPP. There is very little left out: they share their design thinking and coding philosophy on every step in the process. It absolutely reflects real life experience. For example, the very first smoke-test seems quite modest in the beginning, yet reveals more unknowns then initially perceived and we are reminded once again how setting up new applications always takes longer than expected.

It’s as if you’re invited to see how senior engineers from Thoughtworks design and implement a system. However, you will not find authors preaching what is perfect code through artifically prepared examples. The problem is real enough - with external dependencies, asynchronous code and UI challenges. Solutions do not seem to be presented in hindsight; we learn and refactor along with the authors. I think this book will especially help those who, after having absorbed yet another talk, book or blog post about software design or testing, still find themselves slightly lost and producing suboptimal code at work.

The writing style is similar to other books in the Addison-Wesley signature series: rigorous, elaborate and clear. Closing chapters such as those on test readability or testing asynchronous code add even more value. Personally I have found myself sharpening my instinct for recognizing where really is a boundary of responsibility for classes. No matter how nice your programming language or framework is, do not hesitate to create new classes. Also, it helps to pay attention to little details such as not passing special strings / hashes but constants / value types. Maintaining your tests (and their setups) brief, informative and effective will take your project a long way.

Update 2013-07-18: Clarified sentence about writing style after feedback from the authors.

Upcoming platform update

This Sunday (July 7th) we will roll out an update to our build platform. It will include the latest versions of Ruby, Node.js and a few other packages.

Selenium users, Firefox will go up to version 22 so please remember to upgrade your selenium-webdriver gem.

Please see the changelog for the full list of changes.

Testing Rails apps that use Pusher with pusher-fake

Pusher is a great service for sending messages from anywhere to your users’ web browsers. It has all the good features of a cloud service: reliable uptime, great customer service and it enables you to outsource a complicated but probably not critical part of infrastructure.

Within Semaphore we’re using Pusher to send various status updates. For example, our build page has many moving parts but they’re clustered in a few page states. When a build starts, a background job sends a signal to all connected browsers to silently reload the page using Turbolinks and initialize new behaviour.

With an external API doing an important job, testing your system can be tricky. The common solution is to fake the related component with stubbed methods. For example, to test our GitHub integration, we use VCR to store and play back real responses. For Pusher we’d like a different kind of faking: ideally we need to keep the signal going through our backend to the browser but over localhost.

pusher-fake is a fake Pusher server that lets you have just that. The only change to your code is to conditionally load its’ JavaScript in test environment:

<script>
  <% if defined?(PusherFake) %>
    Pusher.host    = <%= PusherFake.configuration.socket_host.to_json %>;
    Pusher.ws_port = <%= PusherFake.configuration.socket_port.to_json %>;
  <% end %>
</script>

And replace the Pusher library’s implementation in the Rails app. If you’re using Cucumber:

require "pusher-fake/cucumber"

If not, recreate cucumber.rb in your initializer. Now you’re ready to run scenarios which trigger code that sends and receives messages.

Setting up Jasmine JavaScript tests for continuous integration

If you are writing JavaScript or CoffeeScript and want to practice TDD, probably the best option for you to implement automatic tests is Jasmine. There are a few nice articles to get you started with the framework (like How do I Jasmine: a tutorial and Testing Your JavaScript with Jasmine) and even a book JavaScript Testing with Jasmine.

Unfortunately it is not so obvious how to get your Jasmine test suite up and running on a continuous integration server such as Semaphore. Here we’ll show you how, assuming that you already have a basic Jasmine test suite in place, written in CoffeeScript as part of a Rails 3 app.

There are several ingredients needed to run a Jasmine test suite in CI:

  • Jasmine - BDD testing framework for JavaScript.
  • Jasminerice - handles Rails assets pipeline which makes CoffeeScript testing a breeze.
  • Guard::Jasmine - for headless testing with PhantomJS.
  • PhantomJS - a headless WebKit scriptable with JavaScript or CoffeeScript. Already available on Semaphore.
  • Sinon.JS - for mocking server responses (optional).

Steps

Add Jasminerice and Guard::Jasmine to your project’s Gemfile:

group :development, :test do
  gem "jasmine"
  gem "jasminerice"
  gem "guard-jasmine"
end

Bundle and than bootstrap Guard::Jasmine:

guard init jasmine
mkdir -p spec/javascripts
echo -e "#=require application\n#=require_tree ./" > spec/javascripts/spec.js.coffee
echo -e "/*\n *=require application\n */" > spec/javascripts/spec.css

With everything in place you are ready to run your tests on a CI service with guard-jasmine command. Semaphore will include test results in a report:

Visit the tools’ pages for more info. There are a lot of instructions that cover specific needs that your project might have.

Heroku add-on goes public

The Semaphore add-on for Heroku is officially out of beta. The add-on makes it easy to set up continuous integration (shhh and soon deployment) for your Heroku-hosted Ruby app.

Big thanks to all the people who’ve used and provided feedback on the add-on while it was in beta, and of course Heroku for the opportunity and great collaboration.

Check it out.

Introducing the project timeline

Sometimes you just want to follow a single project. Or you’d like to catch up and see what’s been happenning. Who’s been busy? Who broke the build and when was it fixed?

Today we’re rolling out project pages with a timeline of all activities at a glance.

The project page begins with a list of its branches, just like on the dashboard. It is followed by the timeline, showing chronologically who pushed what to which branch and the corresponding status. Updates about new and removed branches are included. Everything is live updated. It is a great way to follow your team’s progress.

You can access the project page now from the dashboard, branch and build pages. We’ve been test driving this feature for a while, so you’ll see a nice bit of history of your projects already. We hope you’ll enjoy this feature as much as we do.

ProjectMonitor, BuildHeroes and Coveralls integrate with Semaphore

Here’s a quick roundup of 3rd party tools that let you get more out of Semaphore:

ProjectMonitor is a CI display aggregator for big screens or TVs. The idea is to see the status of all your projects’ builds in highly visible fashion. It’s an open source app that you can run on your own server.

BuildHeroes provides build leaderboards and statistics for your team. After you set up a post-build webhook, it will calculate your build success rate, keep track of developer scores and more.

Coveralls provides code coverage as a service, from repository-wide overviews to line by line data. It can also post a coverage summary to your pull requests on GitHub.

If you’re also making your app or library work with Semaphore by using our API, let us know. We’ll tell our customers about it on this blog.

Get future posts like this one in your inbox.

Follow us on