Case Study: Par8o — Semaphore
How par8o Sped up Rails CI Builds 8x with Semaphore Boosters
Industry: Enterprise Healthcare
Engineering team: 11-50
Stack Ruby on Rails, AngularJS, Elixir, Node.js
- ✓ 8x faster CI with Semaphore Boosters
- ✓ Fully automated parallel CI
- ✓ Higher developer productivity and team engagement
Par8o is a platform designed for healthcare teams to coordinate, communicate, and close the loop on patient referrals — for better relationships with providers and patients. par8o also offers a chat platform to encourage office-to-office and referral communication for quicker results.
The referral portion of par8o is a large application built with Ruby on Rails and AngularJS. It is configurable to support the workflow needs of different organizations.
The engineering team at par8o takes testing seriously and uses MiniTest for unit and controller tests and RSpec for browser testing. They use a development workflow that is very similar to the GitHub workflow, but instead of merging to master after a story has been verified in production, they merge to master once a story has been verified in the QA environment.
At one point, running the tests on development laptops started taking the par8o developers 2 or more hours. As more tests were added, the time increased even further. Par8o started using Semaphore’s parallel job feature to distribute tests across multiple test builders. The goal was to have test feedback for a branch in less than an hour. Using parallel jobs, they were able to run all of the tests in around 50 minutes.
Even though they had met the goal, the team wanted to have even quicker test feedback to avoid context switching back to a branch if the build failed 50 minutes later. The team was hopeful that over time, by manually parallelizing tests, they would bring the build time to around 25 minutes.
Kendal Miller, Senior Engineer at Par8o
“Using parallel jobs, we organized testing files into folders that were loosely based on feature and were able to run all of the tests in around 50 minutes total. While we had met our goal, we wanted to have quicker test feedback for our branches so that we would not need to context switch back to a branch if the build failed 50 minutes later.”
At some point, they managed to have all tests running in around 30 minutes. However, manually organizing the tests into folders was tedious, because naming the folder in a way that made sense to other developers was hard. They were then required to merge the branch containing the new folder organization into all of the other branches currently in development. Finally, they would add a new job in Semaphore to run the tests in the new folder(s). There was also the risk of adding a new test folder, but not adding the job to run tests for that folder, which would result in a build not running all of the tests. If a new folder was added to try to prevent this, they had a test that failed with an error.
So, even though the testing time was drastically reduced, manually setting up parallel testing was tedious and time consuming. Enter Semaphore Boosters.
Faster, Fully Automated Parallel Testing
As par8o team has been using Semaphore, they’ve interacted with Semaphore support to request new features or updates to packages used by the builders. At some point, Semaphore support offered a solution to reducing par8o’s build time that was averaging 50 minutes at the time.
The solution was Semaphore Boosters — a Ruby gem that records test times for a given file and, using this timing information, splits up the tests into multiple commands based on how many jobs you want to run. The goal is for each job to take approximately the same amount of time to complete. Semaphore currently supports Boosters for RSpec and Cucumber tests.
The manual parallelization strategy was working, but it was fragile and required periodic reorganization of test files as a job began to take too much time to complete. par8o’s slowest tests were the RSpec browser tests, and since Boosters is available for RSpec, they decided to try using them.
Improved Engineering Culture
Boosters are easy to set up. All you need to do is go to the bottom of the project page, and click “Go to Boosters”. You can then choose to Add RSpec Boosters. Choose a recent branch and click start.
After adding Boosters, par8o’s build time for the entire test suite averages around 13 minutes. Boosters is letting the par8o developers get results for their test suite faster, and they’re also saving time they previously spent moving files around to spread tests across builders.
Getting faster feedback from tests has made it less likely that developers will need to switch to another task while waiting on tests. If the build fails, they can make the necessary fixes more quickly, while the changes are still fresh in their mind. This saves us time and mental energy, while allowing developers to focus on completing the initial task.
With very little effort, the par8o team was able to parallelize RSpec browser tests so that the average run time dropped from over 2 hours to 13 minutes. If you’re using RSpec or cucumber for your tests, give Boosters a try today. Spending 15 minutes setting up Boosters, most of which is waiting for the build to complete, has been a major time saver for the par8o team.