Since the launch of the flaky tests in open beta, we’ve been working hard on polishing the interface and improving the detection heuristics. Today, we’re excited to announce that the flaky test dashboard is available to all Semaphore users.
So, What are Flaky Tests?
Flaky tests are tests that oscillate between failing and passing without any clear reason. They tend to sneak up on you at the worst possible time, introducing unforeseen delays and making life harder for everyone.
The best place to spot flaky tests is in your CI/CD pipelines. But identifying them used to require a lot of manual work. Now, you can sit down and let Semaphore do that for you.
How Does It Work?
Once this feature is enabled you will find the new dashboard in your project. Here, you can see the flaky tests in your codebase, apply filters to the data, and take action on your failed tests.
You can see the flaky test UI in action below:
How Do I Set It Up?
To enable this feature, you will need to configure test reports in your project. This allows Semaphore to collect and aggregate information on your tests. Once this is done, the flaky test dashboard will be automatically populated as test data is processed.
For more details, check out the flaky test’s docs.
What Changed Since the Beta?
If you’ve been trying out this feature in beta, thank you! Your feedback is pure gold to us. Since the beta, the following things have changed:
- The dashboard filters are populated with sane defaults when flaky tests are first set up.
- Notifications are now available. You can now get notified when new flaky tests appear in your suite.
Try It Out Now
Sign in to your Semaphore account and check out this new feature. We hope it helps you to combat the unpredictability of flaky tests in your projects.
Read more:
- What is a Flaky Test? How to Fix Flaky Tests?
- Best Practices for Identifying and Mitigating Flaky Tests
- End-to-end Flaky test management on Cypress, Playwright and Selenium
