Introducing Semaphore Insights: Find Test Files That Slow You Down
Today we’re happy to announce Semaphore Insights, a new part of Semaphore with the goal to give you useful, actionable feedback on your continuous integration process.
As developers we all want our builds to finish fast, but sometimes we introduce bottlenecks to the test suite without realizing it. This can be by writing tests that make excessive database calls or similar inefficient use of resources. Just like we understand the need for refactoring the application code, we need to find time to refactor and improve our test code too.
So the first available feature in Semaphore Insights is a report on your slowest test files. It displays all test files which take more than a minute to run, and if none such are found then the slowest five by default.
We have found that a minute is a sensible threshold in large projects — more than that is usually a smell and we recommend looking into the code or considering to split the test file. The benefits are both a faster feedback loop your development workflow and parallel execution in CI. For example, if you have a test file that takes 4 minutes to complete, then your whole build cannot complete sooner than 4 minutes, plus some setup and teardown operations that may exist for your project.
How to enable Semaphore Insights
On your project page, click on "Find slowest test files" link and then enable the feature for your test framework. That’s it — the report will be ready after your next successful build. To see the chart of historical performance, just click on a test file to uncover it.
In order to gather the necessary information, Semaphore inserts some additional formatting options to the corresponding test framework’s configuration file, such as .rspec. That should not affect your build in any way, but if you think it does, report it through a support message and we’ll investigate.
We will be rolling out this feature to all users over the next couple of days.
Semaphore Insights currently supports RSpec 3 and Cucumber. If your testing framework is not supported but you would like it to be, we want to hear from you.
We hope that you will find Semaphore Insights useful. If you have any feedback on it, or even have ideas where it could go next, please get in touch.