Tests fail on Semaphore but pass locally
Headless, clean CI environment may make some tests more vulnerable:
- Double-check if your local development environment has some configuration which was introduced manually and is not available after a fresh repository checkout.
- Check if all your testing gems and configuration are up to date.
sleep()is a common source of flaky tests.
- Having your tests depend on a certain order of records will probably not work.
- Tests depending on external web services will likely fail at some point – when that service blocks repeated access or becomes unavailable; use VCR to record responses once and play them back every next time
- Avoid mixing mocks with real records.
- Check if your tests are relying on specific domain configuration in
/etc/hosts. We can add special configuration for you but in general your tests shouldn’t rely on such deep system configuration changes.
- Make sure that your code is not prone to race conditions.
Ruby and Rails specific
- If using Timecop, remember to reset any changes to time.
- Change the database_cleaner strategy to truncation for specs with
:js => true.
- With Rails, the
Timeobjects will ignore the configured Rails timezone in methods like
Time.nowand actually return the system time. You should use methods like
- Increasing Capybara timeout value sometimes helps.