Semaphore Blog

News and updates from your friendly continuous integration and deployment service.

Meteor.js: Getting Started

Get future posts like this one in your inbox.

Follow us on

Meteor.js tutorial with meteor.js tests.

MeteorJS stands out among the recent big Javascript frameworks with its radical view on creating web applications. It seamlessly connects the client side code with a Node + MongoDB based backend and promises unbelievably fast application development.

As an introduction to Meteor.js, I set out to build a simple but properly tested web application and share what I learned.

Installing the Meteor.js framework

Meteor has a one line install process based on a script from its website:

$ curl https://install.meteor.com | /bin/sh

The above command installs all the needed dependencies along with a MongoDB database. After this you are ready to start writing applications.

Creating an example Todo application

$ meteor create todos

Right after the above command finishes you can launch a working web application with the following:

$ cd todos
$ meteor --port 3000
=> Meteor server running on: http://localhost:3000/

Meteor.js tutorial initial screen.

Unit tests

Meteor works with various testing frameworks and test runners. One of the most popular test runners is Velocity that supports Javascript test frameworks like Jasmine and Mocha.

Jasmine — my favorite Javascript test environment — is easy to set up with Meteor and Velocity.

$ meteor add sanjo:jasmine

In Meteor you should put all your tests in a tests directory. Before you start writing your own tests, first create a directory structure like the following one, as required by sanjo:jasmine.

$ mkdir -p tests/jasmine/server/unit
$ mkdir -p tests/jasmine/client/integration

Velocity also offers a nice HTML reporter that overlays your application and shows a red/gray circle in the upper right corner of the application.

Meteor.js velocity tests.

When you click on the circle it will list all the applications test statuses.

Meteor.js test run.

You can install it with the following command:

$ meteor add velocity:html-reporter

Unfortunatelly, in the moment of writing this blog post, there is no standard way to run Velocity tests from the command line. A Node package velocity-cli is trying to change that, at least until there is no meteor test command.

Integration and acceptance tests

For integration tests you can use Nightwatch which will run Selenium-based browser tests on your application. To install it for your application, run:

$ meteor add clinical:nightwatch

Unfortunately, the setup for the rest of the framework is a little painful. First you need to install npm and node, and java runtime.

$ sudo apt-get install nodejs
$ sudo apt-get install openjdk-7

After which you should execute the following installation steps for nightwatch.

$ ln -s .meteor/local/build/programs/server/assets/packages/clinical_nightwatch/launch_nightwatch_from_app_root.sh run_nightwatch.sh
$ sudo chmod +x run_nightwatch.sh
$ sudo ./run_nightwatch.sh

Your integration tests should be placed inside tests/nightwatch directory.

Writing a todo application

To demonstrate the power and expressiveness MeteorJS, I will walk you through the design process of a simple Todo application. This application will have only two features. It will show all todos a user saved in the database, and it will allow us two add more todos.

The server and the client in a MeteorJS application are interconnected, and as a matter of fact, can be even defined in the same file. Let’s create a file named todos.js in the projects root and initialize a Mongo model that can store todos for our application.

Todos = new Mongo.Collection("todos")

In order to handle and check the validity of newly created todos, a class that represents a Todo can come quite handy. Right under the definition of the Todos collection we can create a Todo class.

function Todo(name) {
  this.name = name;
}

Todo.prototype = {
  valid: function() {
    return this.name && this.name != "";
  },

  save: function() {
    Todos.insert({name: this.name});
  }
};

As a good TDD practitioner we should also create a matching test using Jasmine. We can define a test inside the tests/jasmine/server/unit/todo_spec.js file. At the moment we’re still curious and learning, hence implementation first.

describe("Todo", function() {

  beforeEach(function() {
    this.name = "wash dishes"
    this.todo = new Todo(this.name);
  });

  it("accepts name", function() {
    expect(this.todo.name).toEqual(this.name);
  });

  describe("#save", function() {

    beforeEach(function() {
      spyOn(Todos, "insert");

      this.todo.save();
    });

    it("inserts into the database", function() {
      expect(Todos.insert).toHaveBeenCalled();
    });
  });

});

Filling the database on startup

On application startup we can invoke various actions. For example we could prefill our database with Todos if there is no todo defined.

Put the following in the todos.js file.

function prefillTodos() {
  if(Todos.find().count() !== 0) return;

  var todos = ["Create a Todo", "Learn MeteorJS"];

  todos.forEach(function(todo) { new Todo(todo).save(); });
}

if (Meteor.isServer) Meteor.startup(prefillTodos);

And its tests in the tests/jasmine/server/unit/prefill_todos_spec.js file.

describe("prefillTodos", function() {

  beforeEach(function() {
    spyOn(Todos, "insert");
    spyOn(Todos, "find").and.returnValue({count: function() { return 0; }})

    prefillTodos();
  });

  it("initializes the database with default todos", function() {
    expect(Todos.insert).toHaveBeenCalledWith({name: "Create a Todo"});
    expect(Todos.insert).toHaveBeenCalledWith({name: "Learn MeteorJS"});
  });

});

Templates

MeteorJS uses templates to display information to the user. To create a HTML view for the todos, first create a todos.html in the project root and create a todo view like the following.

<head>
  <title>todo_app</title>
</head>

<body>
  <h1>Welcome to Meteor!</h1>

  {{> todos}}
</body>

<template name="todos">
  {{#each todos}}
    {{> todo}}
  {{/each}}
</template>

<template name="todo">
  <p>{{name}}</p>
</template>

This file contains regular HTML notation, spiced up with some curly braces. These curly braces are Handlebars-like constructs and allow us to dynamically insert data in our Html views.

Each template can receive arbitrary JSON data that can be used to parameterize the output of that template. To invoke the rendering of a template MeteorJS uses the {{> templateName}} notation.

Each of these templates can have a helper or an event handler.

Helpers are handy methods that we can invoke inside the definition of our templates. Event handlers let us react to various Javascript events. The following code segment shows how to set up a helper method for our todos template.

Append the following snippet to the todos.js file.

function setUpTodoTemplates() {
  Template.todos.helpers({
    todos: function () {
      return Todos.find();
    }
  });
}

if (Meteor.isClient) setUpTodoTemplates();

Testing templates

In MeteorJS there is a custom of testing templates with Jasmine. We should start with testing our todos template in the tests/jasmine/client/integration/todos_template_spec.js file.

describe("TodosTemplate", function() {

  beforeEach(function() {
    var div = document.createElement("div");
    var data = [{name: "test1"}, {name: "test2"}];

    var comp = Blaze.renderWithData(Template.todos, data);

    Blaze.insert(comp, div);
  });

  it("contains all the passed todos", function() {
    expect($(div).children().length()).toEqual(2)
  });

});

Creating new todos

Combining the above knowledge, we can easily extend our todo application with a form that creates new todos.

We should start with a template in our todos.html file.

<template name="createTodo">
  <input id="newTodo" type="text">
  <button>Save</button>
</template>

Now, insert a template render invocation inside our todos template.

<template name="todos">
  {{#each todos}}
    {{> todo}}
  {{/each}}
  {{> createTodo}}
</template>

To handle button clicks from out createTodo template, we should add an event handler inside the setUpTodoTemplates. The following code snippet registers an event handler that will create a new Todo.

Put the snippet inside the setUpTodoTemplates in the todos.js file.

Template.createTodo.events({
  'click button': function() {
    var input = $("#newTodo");
    var name = input.val();

    input.val("")

    new Todo(name).save();
  }
});

An acceptance scenario for the above action could be written as the following inside a tests/nightwatch/creatingTodos.js file.

module.exports = {
  "Adding a Todo" : function (client) {
    client
      .url("http://localhost:3000")
      .waitForElementVisible("body", 1000)
      .pause(10000)
      .assert.visible("input[type=text]")
      .assert.visible("button")
      .setValue("input[type=text]", "Test Todo")
      .click("button")
      .assert.containsText("body", "Test Todo")
      .end();
  }
};

To run this end-to-end test run the following.

$ sudo bash run_nightwatch.sh

Final screenshot and source code

Meteor.js finished todo application.

You can clone the source code for the whole todo application from my Github repository.

Summary

Meteor is an exciting new technology that allows us to create eventful application with ease. The Meteor platform seems to be quite stable, however the testing libraries seem to be lagging behind and there is a lack of good documentation.

I encountered several stumbling blocks while working with tests, when it was hard to find any information on the issue. For example incompatibilities between Nightwach and the version of Firefox I was using, running integration tests for the client side sometimes unexpectedly failed with an error or exception. Fortunately each of the test libraries are under heavy development and I am expecting high level of test integration in the near future.

Finally, here is a list of articles that helped me understand Meteor better.

Platform update on November 24th

The upcoming platform update is scheduled for Monday, November 24th, 2014.

Chrome driver is updated to version 2.12.

Git is now on version 2.1.3.

PHP updates include versions 5.4.34, 5.5.18, 5.6.2.

RethinkDB is updated to version 1.15.2.

Ruby gets two updates with 1.9.3p551 and 2.0.0p598.

New things

Ruby 2.1.5 is added to the platform. Read more about this version in the Ruby changelog.

Extending support for Ubuntu 12.04

After reading your letters, we decided that our support for Ubuntu 12.04 is going to be extended. Many servers are still running Precise and we really don’t want to rush anyone to do the switch. Especially in the holiday season when there are more important matters to deal with.

We are extending support for Ubuntu 12.04 LTS v1409 platform until June 1st, 2015.

Release candidate available now

You can try out this update right away by selecting version 1411 (release candidate) in the Platform section of Project Settings. To use Ruby 2.1.5, add the following build command rbenv global 2.1.5. In the final release, it will be avaliable through the interface as usual. As always, we’re eager to hear your feedback and thoughts.

Updates for the final release

PhantomJS is now on version 1.9.8 with an important security update for the notorious POODLE vulnerability.

Chrome has been updated to version 39.0.2171.85.

Some users have reported ocassional issues during apt-get update, so /etc/apt/servers.list is updated with new servers which will hopefully be more stable.

Updated libmysqlclient library

An updated version of libmysqlclient-dev is also included, and is now on version 5.6.21. Projects using the mysql2 gem, version 0.3.11 or older, are required to update the gem to at least version 0.3.12 because otherwise the gem can’t be compiled during bundle install.

For this reason we have postponed the platform update to Tuesday, November 25th.

If you experience any issues, please contact us immediately so that we can fix them as soon as possible.

A full list of changes is available in the platform changelog.

BDD on Rails with Minitest, Part 2: Implementing a Feature

BDD on Rails with Minitest tutorial, part 2

Chris Kottom, guest author on Semaphore blog

This is a guest blog post from Chris Kottom, a longtime Ruby and Rails developer, freelancer, proud dad, and maker of the best chili con carne to be found in Prague. He’s currently working day and night on the upcoming e-book The Minitest Cookbook.


A lot of Rubyists and Rails developers use a behavior-driven (BDD) workflow to give structure to feature development. Adoption of BDD using tools like RSpec and Cucumber has been fueled in part by the availability of high-quality books and blog posts on the subject, but RSpec is certainly not the only game in town. Many programmers have begun looking for alternative approaches, for example, based on the Minitest unit testing library which is included in Ruby distributions and provides similar functionality in a more compact implementation.

In this post, we’ll walk through a complete feature implementation and show how to build realistic functionality into our application using our tests to lead us through the process. If you haven’t read the first part of the series where we described how to assemble and get started using a lean and mean Rails testing stack based on Minitest, you should consult that post first since the examples in this post pick up where we left off.

Feature: Display a List of To-Do Items

In part one, we implemented an empty list of to-do items. Now we’re ready to display a full list of items from the database. To do this, we’ll update the tests to verify that all to-do items are shown in the list and that the name and description of each item is displayed. The updated feature to look like this:

require "test_helper"

feature "To Do List" do
  scenario "displays a list of to-do items" do
    visit root_path
    page.must_have_css("#items")
    within("#items") do
      Item.find_each do |item|
        selector = "#item-#{ item.id }"
        page.must_have_css(selector)
        within(selector) do
          page.must_have_content item.name
          page.must_have_content item.description
        end
      end
    end
  end
end

Once we’ve done that, we re-run our suite and see that our feature spec is once again failing.

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.06s)
  test_0002_responds with success                                PASS (0.01s)

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.04s)
NameError:         NameError: uninitialized constant Item
        test/features/to_do_list_test.rb:8:in `block (3 levels) in <top (required)>'
        test/features/to_do_list_test.rb:7:in `block (2 levels) in <top (required)>'
        test/features/to_do_list_test.rb:8:in `block (3 levels) in <top (required)>'
        test/features/to_do_list_test.rb:7:in `block (2 levels) in <top (required)>'

Finished in 0.12654s
3 tests, 3 assertions, 0 failures, 1 errors, 0 skips

In strict BDD, we’d want to step through this very slowly and one step at a time, but to speed things along and keep from having to write separate migrations for each and every attribute separately, we’ll collapse what we currently know about to-do list items and their attributes into a single failing test case.

require "test_helper"

describe "Item" do
  before do
    @item = Item.new(name: "Write Minitest-BDD post",
                     description: "Show Rails and Capybara example")
  end

  it "has a name attribute" do
    @item.must_respond_to :name
  end

  it "has a description attribute" do
    @item.must_respond_to :description
  end
end

The test syntax you see here is provided by Minitest::Spec which provides a declarative API for describing the expected state of objects. In this example, I can call #must_respond_to on any object with a Symbol or String argument to test whether or not the object responds to the named method. The Minitest::Spec DSL is very compact by itself, but it’s trivial to extend t with your own custom expectations that will improve the readability and expressiveness of your tests.

These new model tests will fail for the same reason as the controller and feature tests: there’s no Item model yet defined.

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.06s)
  test_0002_responds with success                                PASS (0.01s)

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.04s)
NameError:         NameError: uninitialized constant Item
        test/features/to_do_list_test.rb:8:in `block (3 levels) in <top (required)>'
        test/features/to_do_list_test.rb:7:in `block (2 levels) in <top (required)>'
        test/features/to_do_list_test.rb:8:in `block (3 levels) in <top (required)>'
        test/features/to_do_list_test.rb:7:in `block (2 levels) in <top (required)>'

Item
  test_0002_has a description attribute                          ERROR (0.00s)
NameError:         NameError: uninitialized constant Item
        test/models/item_test.rb:5:in `block (2 levels) in <top (required)>'
        test/models/item_test.rb:5:in `block (2 levels) in <top (required)>'

  test_0001_has a name attribute                                 ERROR (0.00s)
NameError:         NameError: uninitialized constant Item
        test/models/item_test.rb:5:in `block (2 levels) in <top (required)>'
        test/models/item_test.rb:5:in `block (2 levels) in <top (required)>'


Finished in 0.13013s
5 tests, 3 assertions, 0 failures, 3 errors, 0 skips

All of the current errors will be resolved by generating a new Item model with :name and :description attributes and running migrations. Our three errors collapse into one failure.

Item
  test_0001_has a name attribute                                 PASS (0.01s)
  test_0002_has a description attribute                          PASS (0.00s)

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.30s)
  test_0002_responds with success                                PASS (0.01s)

To Do List Feature Test
  test_0001_displays a list of to-do items                       FAIL (0.05s)
Minitest::Assertion:         expected to find #item-298486374.
        test/features/to_do_list_test.rb:10:in `block (4 levels) in <top (required)>'
        test/features/to_do_list_test.rb:8:in `block (3 levels) in <top (required)>'
        test/features/to_do_list_test.rb:7:in `block (2 levels) in <top (required)>'

Finished in 0.38305s
5 tests, 6 assertions, 1 failures, 0 errors, 0 skips

We now need to modify the view to display our list of items, and just to speed things up here, we’ll break with strict BDD and once again combine individual changes to the view to include the name and description of the item as well as the id selector that we’ll need. In reality, our view might end up looking like this:

<%= content_tag :div, class: 'items' do %>
  <% @items.each do |item| %>
    <%= content_tag :p, id: "item-#{ item.id }", class: 'item' do %>
    <%= content_tag :b, item.name %>
      <br>
      <%= item.description %>
    <% end -%>
  <% end -%>
<% end -%>

In the end, we end up with errors across all tests that touch the view.

Item
  test_0002_has a description attribute                          PASS (0.00s)
  test_0001_has a name attribute                                 PASS (0.00s)

ItemsController::GET :index
  test_0001_renders items/index                                  ERROR (0.11s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'

  test_0002_responds with success                                ERROR (0.00s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'


To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.01s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___267186296561738095_69977433372580'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'

Finished in 0.14093s
5 tests, 2 assertions, 0 failures, 3 errors, 0 skips

The cause of these failures is the same in all cases: the collection of to-do items is not yet being set by the controller. To resolve this, we first write a failing test on the controller that checks that the collection is assigned and has the right values.

it "fetches and assigns a list of to-do items" do
  assigns(:items).wont_be_nil

  item_ids = assigns(:items).map(&:id).sort
  item_ids.must_equal Item.pluck(:id).sort
end

Here again, we see more of the Minitest::Spec DSL in action. Minitest provides negative expectations like #wont_be_nil to check for the absence of a given condition - in this case, that the object under test is non-nil.

Unlike RSpec, Minitest doesn’t provide a way of testing whether two Arrays contain the same elements, but this is easy enough to implement in the case of ActiveRecord relations simply by comparing two sorted arrays of IDs as shown in the example.

Including this test adds one more failure to our suite:

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.14s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'

Item
  test_0001_has a name attribute                                 PASS (0.00s)
  test_0002_has a description attribute                          PASS (0.00s)

ItemsController::GET :index
  test_0001_renders items/index                                  ERROR (0.00s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'

  test_0002_responds with success                                ERROR (0.00s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'

  test_0003_fetches and assigns a list of to-do items            ERROR (0.00s)
ActionView::Template::Error:         ActionView::Template::Error: undefined method `each' for nil:NilClass
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'
        app/views/items/index.html.erb:2:in `_app_views_items_index_html_erb___1343446998779429432_69831990148440'
        test/controllers/items_controller_test.rb:6:in `block (3 levels) in <top (required)>'

Finished in 0.15893s
6 tests, 2 assertions, 0 failures, 4 errors, 0 skips

And now we only need to update the controller action to make the tests pass:

class ItemsController < ApplicationController
  def index
    @items = Item.all
  end
end

And drumroll please…

Item
  test_0001_has a name attribute                                 PASS (0.00s)
  test_0002_has a description attribute                          PASS (0.00s)

To Do List Feature Test
  test_0001_displays a list of to-do items                       PASS (0.15s)

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.00s)
  test_0002_responds with success                                PASS (0.00s)
  test_0003_fetches and assigns a list of to-do items            PASS (0.00s)

Finished in 0.16973s
6 tests, 13 assertions, 0 failures, 0 errors, 0 skips

This simple example only scratches the surface of what Minitest can do, of course, but it shows off its status as a viable alternative to RSpec. Minitest has a strong and growing ecosystem surrounding it that makes further customization and extension possible to suit your workflow - whether that’s BDD or any other. If you’re looking for a slim alternative to RSpec, it might be worth checking out.

For more information about Minitest and the related gems used in this post, check out some of the following resources:

Ruby 2.1.4 Available In a Minor Platform Update

We just released a minor platform update — v1410.1.

Ruby 2.1.4 is added. This is a security release recommended for all projects using 2.1.

Qt is now downgraded to version 4.8.1 due to the infamous Errno::EPIPE: Broken pipe error that many of you using capybara-webkit have encountered since the last update.

OpenJDK is now on version 1.7 update 71.

Cassandra is updated to 2.1.1.

A full list of changes is available in the platform changelog.

Build queue issues

On Thursday, October 23rd after the rollout of a planned platform update at 13:30 UTC, Semaphore experienced issues which caused delays in running builds and deploys, coupled with decreased performance. First, we want to apologize for that. We know it messed up your workdays. That’s not how we want to do things and will do better next time. Second, I’d like to take a moment to explain what happenned.

Build errors after platform update

More than a few projects experienced unexpected build failures related to mysql2 gem. This was caused by the fact that in this platform update we migrated all projects to a new version of OS, Ubuntu 14.04 with different system libraries. Since Semaphore caches your project’s git repository and installed dependencies between builds, there have been cases where dependencies such as Ruby gems depending on system libraries could not work.

While we did our best to help everyone as soon as they raised the issue — on support, live chat or Twitter — it was also not the way we intended things to go. Our goal is that you don’t have to be aware of such details and not have unexpected failures which require an action on our end.

For this reason, we immediately shipped a small tool to clear a project’s dependency cache, now available in project settings. And at 17:02 UTC we cleared the cache for all projects. This resulted in new git clones and dependency installations for all, but without unexpected failures. We will be evaluating how to do this more granularly when a similar major update comes along next time.

Slow build queue

At the same time, our infrastructure was experiencing a larger issue where machines would go down at a very high rate and frequency. While we have a system to automatically detect this situation and reschedule any jobs that were running on a machine that’s affected, it couldn’t solve the problem completely because the failures were happening too fast, adding up more and more jobs to our build queue. We considered increasing our capacity but realized that it would not remedy the problem.

At 18:50 UTC we identified a memory leak in use of Java-related services, such as Solr and Cassandra, which was causing the failure of build servers. After some consideration we settled on the first guess that it was caused by platform update’s switch to Oracle JVM as the default JVM, and by 21:46 UTC we shipped a revert back to OpenJDK globally.

It took some more time to become evident that this was not a solution, and eventually we realized that the only change left was a memory limiter on LXC containers that run the builds, causing unexpected behaviour when certain Java processes hit it. We reverted this implementation of memory limiting at 22:10 UTC and all builds were able to start and finish normally. At 22:59 the build queue was clear (as we announced on Twitter) and new builds were starting at normal speed.

What we’ll do to avoid this in the future

While we do extensively test every platform update before the final release, it is not possible to recreate every possible scenario that comes from real world usage of the service. For this reason our current plan is to extend our infrastructure to securely run copies of a fraction of jobs with a new version of the platform.

Once again we would like to apologize to you for an interruption in your work. We know how Semaphore’s CI service is important to you and while this is our first situation worthy of a post-mortem in more than two years, we see failures as inevitable. It is our imperative however to shield you from them as much as we can.

BDD on Rails with Minitest, Part 1: Up and Running

BDD on Rails with Minitest tutorial

Chris Kottom, guest author on Semaphore blog

This is a guest blog post from Chris Kottom, a longtime Ruby and Rails developer, freelancer, proud dad, and maker of the best chili con carne to be found in Prague. He’s currently working day and night on the upcoming e-book The Minitest Cookbook.


Behavior-driven development (BDD) has gained mindshare within the Ruby and Rails communities in no small part because of a full-featured set of tools that enables development guided by tests at many different levels of the application. The early leaders in the field have been RSpec and Cucumber which have each attracted millions of users and dozens if not hundreds of contributors. But a growing number of Rubyists have started to build and use testing stacks based on Minitest which provides a comparable unit testing feature set in a simplified, slimmed-down package.

In this two-part series, I’ll show you how to implement a BDD workflow based on Minitest and hopefully introduce you to an alternative means for getting the same behavior-driven goodness into your application in the process. In this post, you’ll see how to set up your testing stack and run through a quick iteration to verify that everything is configured and working properly.

Setting Up

As an example, we’ll work on a simple Rails application that lists to-do items consisting of a name and a description initially and build that BDD-style using Minitest and other supporting tools. Before writing the first test, we need to install and configure our testing stack. In this case, we’re going with a combination of gems that will provide an experience that should be familiar to regular RSpec users:

To install the required gems, we need to add the following lines to the Gemfile.

group :test do
  gem 'minitest-rails-capybara'
  gem 'minitest-reporters'
end

Minitest supports two different methods for writing tests: a default assert-style syntax which resembles class Test::Unit and a spec-style syntax that more closely resembles RSpec. I personally prefer the spec-style syntax, so for this example, we’ll be writing all the tests as specs. To force the generators to produce spec-style test cases, we need to tell them to do so by adding the following block to config/application.rb:

# Use Minitest for generating new tests.
config.generators do |g|
  g.test_framework :minitest, spec: true
end

Next, we’ll update our test_helper.rb by running the minitest-rails generator: rails generate minitest:install. The new version of the test helper requires minitest-rails as a basis for all tests. We’ll modify that a bit further so that the final version also requires rails-minitest-capybara and minitest-reporters and configures the reporters. The finished product should look something like this:

ENV["RAILS_ENV"] = "test"
require File.expand_path("../../config/environment", __FILE__)
require "rails/test_help"
require "minitest/rails"
require "minitest/rails/capybara"

require "minitest/reporters"
Minitest::Reporters.use!(
  Minitest::Reporters::SpecReporter.new,
  ENV,
  Minitest.backtrace_filter
)
class ActiveSupport::TestCase
  ActiveRecord::Migration.check_pending!
  fixtures :all
end

A Failing Feature

The most basic feature of our to-do list will be a page that displays a list containing our to-do items, so we’ll start by generating a new Capybara feature using the provided generator: rails generate minitest:feature ToDoList. That initializes a boilerplate test in test/features that we can then update to suit our needs. We’ll begin by writing the most basic possible test for this application with minitest-rails-capybara providing a nice bridge between two worlds - packaging the Capybara DSL as Minitest-style assertions and expectations.

require "test_helper"

feature "To Do List" do
  scenario "displays a list of to-do items" do
    visit root_path
    page.must_have_css("#items")
  end
end

When I run my test suite, it bombs. Predictably.

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.00s)
NameError:         NameError: undefined local variable or method `root_path' for #<#<Class:0x007f83c41efee8>:0x007f83c4153930>
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'

Finished in 0.00508s
1 tests, 0 assertions, 0 failures, 1 errors, 0 skips

This is exactly what we hope to see, of course. Our workflow dictates that we first establish the desired behavior, see that it fails, and then write the code to make it pass. We’re now ready to begin stepping our way through the implementation using the tests as a feedback mechanism.

The Red-Green Dance

The initial error occurs because the application has no routes defined yet, so we’ll need to add a root path pointing to a hypothetical ItemsController to config/routes.rb:

Rails.application.routes.draw do
  root to: 'items#index'
end

When we re-run the tests, the error that occurs has changed.

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.00s)
ActionController::RoutingError:         ActionController::RoutingError: uninitialized constant ItemsController
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'

Finished in 0.00736s
1 tests, 0 assertions, 0 failures, 1 errors, 0 skips

The route we just defined expects an ItemsController with an index action. It would be easy enough to just create or generate this, but our workflow dictates that first we need a failing test describing what that controller action should look like. At the moment, we’re interested in the shortest path to display a view with an empty list of items. That produces a controller test like this:

require "test_helper"

describe "ItemsController" do
  describe "GET :index" do
    before do
      get :index
    end

    it "renders items/index" do
      must_render_template "items/index"
    end

    it "responds with success" do
      must_respond_with :success
    end
  end
end

Most of the heavy lifting for the simple checks you see here is provided by Rails’ ActionController::TestCase with the spec-style expectations provided as syntactic sugar by the minitest-rails gem. Minitest’s spec syntax resembles that of earlier versions of RSpec before recent changes to the way specs are defined and the introduction of the expect syntax. The resulting code is terse but expressive and reads well with no unnecessary noise.

Now when we run tests, we have errors in the feature and the two new controller tests.

To Do List Feature Test
  test_0001_displays a list of to-do items                       ERROR (0.00s)
ActionController::RoutingError:         ActionController::RoutingError: uninitialized constant ItemsController
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'
        test/features/to_do_list_test.rb:5:in `block (2 levels) in <top (required)>'

ItemsController::GET :index
  test_0001_renders items/index                                  ERROR (0.00s)
NameError:         NameError: Unable to resolve controller for ItemsController::GET :index

  test_0002_responds with success                                ERROR (0.00s)
NameError:         NameError: Unable to resolve controller for ItemsController::GET :index

Finished in 0.01033s
3 tests, 0 assertions, 0 failures, 3 errors, 0 skips

Awesome, it’s practically raining errors! Since all of these are caused by the lack of an ItemsController, we can fix them by creating one. We’ll use the Rails generator - making sure not to overwrite the controller test we’ve already started working when prompted. Once that’s done, re-running the test yields the following output.

To Do List Feature Test
  test_0001_displays a list of to-do items                       FAIL (0.02s)
Minitest::Assertion:         expected to find #items.
        test/features/to_do_list_test.rb:6:in `block (2 levels) in <top (required)>'

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.00s)
  test_0002_responds with success                                PASS (0.00s)

Finished in 0.03471s
3 tests, 3 assertions, 1 failures, 0 errors, 0 skips

The generated controller clears up both controller test errors for the time being, and the fact that there’s now a controller action and basic view has fixed the previous error and turned it into a failure. All that needs to happen now is to add an empty list of items to the view code by replacing the boilerplate view with:

<%= content_tag :div, class: 'items' do %>
<% end -%>

And when we run the tests again:

To Do List Feature Test
  test_0001_displays a list of to-do items                       PASS (0.02s)

ItemsController::GET :index
  test_0001_renders items/index                                  PASS (0.00s)
  test_0002_responds with success                                PASS (0.00s)

Finished in 0.03379s
3 tests, 3 assertions, 0 failures, 0 errors, 0 skips

All green! In only a few minutes, we’ve created a new application and driven our way to the first feature from start to finish using tests as a guide.

This gives you a taste of how to get started building a new Rails application using BDD and Minitest. In part two of the series, we’ll dig in deeper and show how to add more realistic functionality to the application while driving the whole process through our tests.

For more information about Minitest and the related gems used in this post, check out some of the following resources:

This tutorial continues in “BDD on Rails with Minitest, Part 2: Implementing a Feature”.

Upcoming Platform Update on October 23rd

The upcoming platform update is scheduled for Wednesday, October 23, 2014.

JRuby is updated to version 1.7.16.

Three PHP version updates are inluded too, namely 5.4.33, 5.5.17 and 5.6.1.

Git is updated to version 2.1.1.

Firefox is running version 33. Every project using the selenium-webdriver gem is required to update it with bundle update selenium-webdriver.

New things

We’ve also prepared quite a few additions to the platfrom based on your requests.

The list of additions is as follows:

You can try out this update right away by selecting version 1410 (release candidate) in the Platform section of Project Settings. As always, we’re eager to hear your feedback and thoughts.

A full list of changes is available in the platform changelog.

Continuous Integration for Bitbucket Projects on Semaphore

Semaphore launches continuous integration for Bitbucket projects

With great pleasure we are announcing that Semaphore is extending its hosted continuous integration and deployment service with support for Bitbucket repositories. Bitbucket is a source code hosting platform for the distributed version control systems (DVCS) Git and Mercurial, made by Atlassian.

As of today, Semaphore has everything you need to automatically test any project from a Git repository on Bitbucket. This includes setting up a full continuous delivery pipeline to any server or cloud provider, including Heroku and AWS.

The service is free for open source projects and up to 100 private builds per month.

Create a free account and start testing your projects today .

Adding your Bitbucket projects to Semaphore

Setting up continuous integration for a Bitbucket project and running your first build on Semaphore is easy and takes only a few clicks.

After you click to add a new project, select Bitbucket as your repository host. Next you will be redirected to Bitbucket to authorize Semaphore to access your projects. Keep in mind that Bitbucket only supports giving access to both private and public repositories. Once you confirm permission access, Semaphore will present you with a list of your Bitbucket repositories.

In the next steps you will select Bitbucket repository, branch to build and the Semaphore account that you want the repository to be attached to. Note that in order to add some repository from Bitbucket, user has to be either its creator or to belong to a Bitbucket group that has admin rights to the specific repository.

After that Semaphore will perform a quick analysis of the source code and generate a set of working build commands depending on your project’s programming language.

At this point you’ll be ready to launch your first build. Here’s what one of our latest looks like:

If you are using Bitbucket, we hope that you are excited about this feature as we are. Happy building!

Deploy from your chat room

Remember how excited you were when Jesse Newland from GitHub presented ChatOps at GitHub? Great news! From now on you can set up your own Hubot to perform deployments on Semaphore.

Several weeks ago Ben Straub from Gridium asked us about a way to trigger a deploy via Semaphore API. We exchanged several mails and as a result he sent us a Hubot script to run a deploy on Semaphore. On the screenshot below you can see it in action as one of the founders from Gridium deploys a project to production right from a Slack chat room.

Deploying to production with Semaphore from a Slack chat room

How to set up Hubot

Hubot — GitHub’s chat bot — is a friendly robot that lives in your chat room and helps you and your team with simple tasks.

It can be deployed on a wide variety of platforms, including Linux and Windows, but a popular option is to deploy it to Heroku. Detailed instructions about installation can be found in Hubot’s Readme file.

After deployment you need to invite the bot in your chat room. Hubot has simple integration with popular chat services such as Slack and Campfire, but it can also work on many other chat services.

Make your Hubot work with Semaphore

In order to extend your own Hubot with the deployment feature, please clone the hubot-semaphore-deploy repository and follow the tutorial in Hubot’s Readme on adding scripts.

After installing the script you will need to copy the API auth token from Semaphore and export it in your Hubot’s production environment. The auth token can be found in your project’s setting under the API tab.

export HUBOT_SEMAPHOREAPP_AUTH_TOKEN=<your-api-auth-token>

And that’s all there is to have deploy power right in your chat room. Having this feature in your company’s workflow can increase awareness of what everyone is working on, and also reduce the time newcomers need to get up to speed with the process of continuous integration and delivery.

Update: Ben Straub joined forces with the author of hubot-semaphoreapp and combined this project with Semaphore status updates.

A Look Inside Development at 500px

In the “Developer Interview” series we talk to developers from some of the companies using Semaphore to find out how they work and share their insights with you. This time we chatted with Devon Noel de Tilly, QA Engineer at 500px, a photo community for discovering, sharing, buying and selling inspiring photography powered by creative people worldwide.

Tell us about your role and responsibilities at 500px.

Devon Noel de Tilly, QA engineer at 500px

I’m a QA Engineer at 500px. I look after our CI setup which runs on Semaphore, maintain our suite of automated tests, run manual tests in a pre-production environment, manage deployment, find and fix bugs, and work closely with our developers and sysadmins. My role is something of a hybrid between testing, automation / DevOps and sometimes development. I work primarily on our web apps and supporting services.

Your products serve an established community of users. What does your workflow typically look like at this point, from an idea until it’s shipped?

A lot of what we do at 500px comes out of ideas from our developers and designers. We hold monthly hack days, and a lot of our new features have come from that, as well as just discussions amongst team members. Of course we get ideas from our users, what they want and what is important to them. And sometimes our executives will come to the team with what they’d like to see.

We hold monthly hack days, and a lot of our new features have come from that.

Usually after someone gets an idea, they’ll make a small proof of concept, often for hack day but sometimes they’ll just pitch it to anyone who’ll be involved in making it a reality. If people like the idea and are on-board for it, then the developers or designers will start working on it. Our technical staff is separated into teams at 500px, so we have a web team, a platform team, an iOS team, etc. that are all largely self-contained, but collaborate closely when necessary. Usually a project will be scoped to one team, with maybe a bit of support from some of the other teams, so planning is generally pretty minimal.

After a team has decided they’d like to do something, and our project managers have split the work up into tickets and slotted it in for a sprint, it’s largely up to the individual teams how they want to proceed. For example, our platform team likes to do things in small bite-sized tickets and work either in pairs or alone, hammering out a new feature a little bit at a time, while our web team prefers to get together in a big room with their laptops and all work on a new feature together.

500px team

Once the initial design and development is done for a ticket, the developer will open a pull request, and it’ll go through code review, where other designers and developers on the team will point out flaws and make suggestions for improvement. After code review, it’ll enter QA, where we’ll run the full suite of automated tests through Semaphore against the branch, as well as do some manual and load tests in a pre-production environment. If there are failed specs, or if we find bugs through manual testing, QA will gather any relevant logs, errors and steps to reproduce, sometimes make suggestions, reject the ticket, and developers will work to fix it. After a ticket has been accepted by QA, we’ll merge it into the master branch and deploy the code to users.

Did you have times when technical debt slowed down this process? If so, how did or do you overcome it?

Yeah, technical debt I think is a problem for any code base once it’s become sufficiently large. We try to be proactive about it as much as possible, but there have been times when it’s come back to bite us a bit, or when its slowed down our development cycle for sure.

It’s not so much a problem we’ve overcome as a problem we’re always overcoming. It’s something we have to be constantly mindful of. How we deal with it is again largely on a team by team, or even individual by individual basis. We use Code Climate to analyze our code base, and that helps us find some things we’ve overlooked. Sometimes members of the team who find themselves with time will spend it combing over the code base and addressing some of our technical debt.

For me, when I started at 500px, a big part of the technical debt we had at the time was the state of our spec suite, which wasn’t really being maintained and was a bit of a mess.

500px team

What was your main guideline in getting to a green build?

I wouldn’t say I really had one main guideline, unless you count the vastness of the internet as a whole. I drew from a variety of resources. But I approached the problem knowing where our weaknesses were (mostly race conditions and inefficiently loading test resources), and keeping best practices in mind, found our pain points and corrected them a little at a time.

I used Relishapp’s Rspec documentation and Betterspecs as a guideline for best practices, as well as a variety of great tech blogs like Airbnb’s blog and Semaphore’s own blog.

There’s an idea that if you care about something, and you want to work on it, then work on it and its yours.

Given the amount of tests in our suite and the amount of race conditions that needed addressing, I also realized early on that I’d have to make a couple compromises. I found a great little gem called respec that we use to rerun failed specs exactly once at the end of our test build, and that helped us a lot in the beginning.

Do you release new features to all users immediately?

Often but not always. Small new features and bug fixes we generally push to all users right away, but some of our bigger features we’ll protect behind rollout flags and rollout internal first, so we can all get acquainted with it and find any problems or things that we might’ve overlooked. If something has the potential to really hammer our API, and we’re not sure how it’ll preform, we’ll sometimes do a staged rollout to percentages of our users also. And we also do A/B tests when we’re unsure how users will respond to a new feature or layout.

500px team

How do you manage the code for your main application on 500px.com? Is it pretty much a classic MVC web app, or you’ve branched out into something more custom?

At one point our main site was basically one huge Rails app (which we call The Monolith internally), but as our platform got bigger it became harder and harder to maintain. So we’ve started splitting everything up into a microservices architecture. The idea is that any logically distinct part of code should be self-contained.

Ruby, and also Rails, are designed primarily to be easy to use and pleasant for the programmer, that’s a big philosophy in the Ruby and Rails community. That things should Just Work. Which is awesome, but once you start getting into big computation on the backend, it can be pretty slow and inefficient. So we started splitting out some of our really slow code, our pain points, into their own Go services, which is much more performant. Things like search and photo conversion.

What’s your favorite part in working at 500px?

There are a lot of things I like about working at 500px. The opportunities to work on things that interest me, and to grow my skills and knowledge. The great benefits and team outings. The people I work with, they really make it a great place to work. But if I had to pick something, it would probably be the attitude towards ownership of the product. The autonomy I’m given is great, I don’t feel like I have people looking over my shoulder and telling me what to do all the time.

At 500px, there’s an idea that if you care about something, and you want to work on it, then work on it and its yours. That we own the things we’re working on, that we’re ultimately responsible for the things we decide are important. Of course there are disagreements sometimes about what we should be working on, of course there’s pressure from other team members to focus on something, but that’s the ideal we’re working towards. And I feel like I can really get behind that message.

Get future posts like this one in your inbox.

Follow us on