Smoke testing is often mentioned but often in various contexts. In this article we explore the idea and how you can apply it in your CI pipeline.
Smoke testing is a technique responsible for discovering major flaws in software early in your continuous delivery pipeline. It consists of very shallow, but broad tests that verify the crucial parts of the application. This makes smoke testing a very cost-effective operation.
In this article, we will explore the origins of smoke testing and how it can be incorporated into your continuous integration or continuos delivery pipeline.
Origins of Smoke in Testing
The term smoke test originally comes from electrical engineering. Once the circuit of an electrical device is ready for testing, it is attached to the power for the first time. If there are some major defects, smoke will appear almost instantly. It’s a clear sign to the engineers that no further tests are needed (or even possible) before the circuit is fixed.
The same concept was adopted in software development, together with the original term.
What and How We Should Test
Smoke tests should determine if the system is stable enough for it to make sense to proceed to the next phase. This is achieved by testing the main functionalities of the system at a very basic level.
For example, if we are working on an e-commerce web application, we would need to check if a customer can purchase a product. We don't need to verify all the details and edge cases of the purchasing process. Passed smoke tests are usually followed by a comprehensive test suite.
Although it is common to smoke test the main user-facing features of an application, you can use them at many different abstractions levels in your entire system. The value of this multi-level smoke testing approach is that you can discover defects on a more controlled and granular level. When working on distributed systems that involve many teams, this can be a big time saver.
Here are a couple more examples of how you can apply smoke testing:
Dependency check - verifies that your application has all dependencies met in order to boot. These could be libraries, network and database connections, etc.
Server probe - after server software upgrades, which can break your production environment in unforeseen ways. With a good smoke test you can easily and cheaply save yourself from introducing broken servers into production.
Post-deploy verification - after a deploy to a server, a smoke test can verify if the deploy has been successful, and initiate automatic rollback if the deploy turns out to be broken.
Depending on what you are testing, actions that you will take based on the result can vary a lot. However, it is essential that they are fast in order to ensure a fast feedback loop.
Smoke tests should evolve as your application evolves. In the beginning, they could check for one or two essential functions, and over time they may need to grow to satisfy their purpose.
Tools that can serve well for writing smoke tests can be very simple. For
instance, even a simple
curl api.myapp.com/is_alive is a good start. Checking
versions of installed dependencies with a simple Bash script could also be
For more complex checks, you can rely on a testing library in your favourite programming language. If you are using Ruby and want to test some of the main features of your web application, you can run a few Cucumber scenarios against your production instance.
Smoke testing is a practice that allows you to catch errors that would be detected anyway, but do so much faster and cheaper. You can start simple by adding one basic test as the first step in your continuous integration pipeline. It takes little and can save a lot of time.
P.S. Would you like to learn how to build sustainable Rails apps and ship more often? We've recently published an ebook covering just that — "Rails Testing Handbook". Learn more and download a free copy.