This is a guest post by Kendal Miller, Senior Engineer at par8o. Read it to discover how par8o reduced build time from over 2 hours to 13 minutes with Semaphore Boosters, fully automated parallel CI for Rails.
par8o is a platform designed for healthcare teams to coordinate, communicate, and close the loop on patient referrals – for better relationships with providers and patients. par8o also offers a chat platform to encourage office-to-office and referral communication for quicker results.
The referral portion of par8o is a large application built with Ruby on Rails and AngularJS. It is configurable to support the workflow needs of different organizations. The application is also used to integrate with other healthcare systems.
The engineering team at par8o takes testing seriously and uses MiniTest for unit and controller tests and RSpec for browser testing. We use a development workflow that is very similar to the GitHub workflow, but instead of merging to master after a story has been verified in production, we merge to master once a story has been verified in our QA environment by our quality assurance team.
Before – Manual Parallel Testing
Prior to using Boosters, par8o was using the parallel job feature of Semaphore to distribute our tests across multiple test builders. Our initial goal was to have test feedback for a branch in less than an hour. Prior to Semaphore, running the tests on our development laptops would take 2 or more hours and cause the laptops to become hot. This time increased as more tests were added. Using parallel jobs, we organized testing files into folders that were loosely based on feature and were able to run all of the tests in around 50 minutes total.
While we had met our goal, we wanted to have quicker test feedback for our branches so that we would not need to context switch back to a branch if the build failed 50 minutes later. We were hopeful that over time, by manually parallelizing our tests, we would bring the build time to around 25 minutes.
At some points, we did have our tests running in around 30 minutes. However, manually organizing the tests into folders was tedious because naming the folder in a way that made sense to other developers was hard. We were then required to merge the branch containing the new folder organization into all of the other branches currently in development. Finally, we would add a new job in Semaphore to run the tests in the new folder(s). There was also the risk of adding a new test folder but not adding the job to run tests for that folder which would result in a build not running all of the tests. (We had a test that failed with an error if a new folder was added to try to prevent this!)
Discovering Faster, Fully Automated Parallel Testing
As we’ve been using Semaphore, we’ve interacted with Semaphore support to request new features or updates to packages that the builders are using. During one of these email threads, Semaphore pointed out that our build time was currently averaging 50 minutes. They also offered a solution – Boosters.
Boosters is a Ruby gem created by Semaphore that records test times for a given file and using this timing information, splits up the tests into multiple commands based on how many jobs you want to run. The goal of this is that each job will take about the same amount of time to complete. Semaphore’s service currently supports Boosters for RSpec and Cucumber tests.
Our previous parallelization strategy was working for us, but it was fragile and required periodic reorganization of test files as a job began to take too much time to complete. Our slowest tests were the RSpec browser tests, and since Boosters is available for RSpec we decided to try Boosters.
Setting Up Boosters
Boosters is very easy to set up. At the bottom of your project page, click “Go to Boosters”. You can then choose to Add RSpec Boosters. Choose a recent branch and click start.
During the initial Booster build, the builders are provided by Semaphore and do not count against your own plan.
Once the build is complete, some timing information will appear with a slider that lets you determine how many parallel jobs Boosters should create.
After adding Boosters, our average build time for the entire test suite is around 13 minutes! And if we were able to speed up some of our MiniTest jobs, the entire suite time would be around 11 minutes. Boosters is letting us get results for our test suite faster, and we’re also saving time because we don’t have to move files around to spread tests across builders like we were doing previously.
Getting faster feedback from the tests has made it less likely that we will need to switch to another task while waiting on tests. If the build fails, we can make the necessary fixes more quickly while the changes are still fresh in our mind. This saves us time and mental energy, while allowing us to focus on completing the initial task.
With very little effort, we were able to parallelize our RSpecs browser tests so that the average run time dropped from 50 to 13 minutes. If you’re using RSpec or Cucumber for your tests, give Boosters a try today. Spending 15 minutes setting up Boosters, most of which is waiting for the build to complete, has been a major time saver for our team.