Post originally published on https://rossta.net. Republished with author’s permission.
A useful feature of RSpec is the ability to pass metadata to tests and suites.
You may already be familiar with how Capybara uses the :js
option to enable the JavaScript driver.
describe "a javascript feature", :js do
# tests run against the Capyabara.javascript_driver
end
Capybara provides an RSpec configuration hook that changes the web driver for any example where :js
metadata is present. Here it is, oversimplified:
# capybara/rspec.rb
RSpec.configure do |config|
config.before do
Capybara.current_driver = Capybara.javascript_driver if example.metadata[:js]
end
end
We may reach a point in the maturity of our test suite when it makes sense to add our own configuration options.
The examples in the post are based on RSpec version ~> 3
.
Changing Test Runner Behavior
Testing libraries like RSpec and Capybara do some heavy lifting to set up the Rails environment and make it suitable for running in test mode. For performance reasons, it may be beneficial to run each of our specs in a database transaction so test data can be easily rolled back at the start of each spec.
Here’s a common base configuration for using the popular DatabaseCleaner gem to set up transactional database behavior for RSpec:
RSpec.configure do |config|
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
Not all specs can be run this way β once we’ve added a JavaScript acceptance spec, for example, the JavaScript driver will likely need its own connection to the database, so it won’t have access to data setup in the tests. We need to run JavaScript acceptance specs in truncation mode to ensure database changes are committed to the database, so multiple database connections will have access to the same data.
Let’s use RSpec metadata to toggle database behavior automatically when using the JavaScript driver (i.e. not the default :rack_test
driver). We’ll add the following hooks, borrowed from the DatabaseCleaner README:
# spec/spec_helper.rb
config.before(:each, type: :feature) do
# :rack_test driver's Rack app under test shares database connection
# with the specs, so continue to use transaction strategy for speed.
driver_shares_db_connection_with_specs = Capybara.current_driver == :rack_test
if !driver_shares_db_connection_with_specs
# Driver is probably for an external browser with an app
# under test that does *not* share a database connection with the
# specs, so use truncation strategy.
DatabaseCleaner.strategy = :truncation
end
end
We also run into problems with ActiveRecord after_commit
callbacks β when running tests in transaction mode, these callbacks will never fire. We can also add an option for enabling truncation mode outside of acceptance specs when isolated specs are needed for these callbacks:
# spec/model/user_spec.rb
it "triggers background job after creating new user", :truncation_mode do
# test after_commit callback
end
# spec/spec_helper.rb
config.before(:each, :truncation_mode) do
DatabaseCleaner.strategy = :truncation
end
Here’s a consolidated configuration for providing hooks for the issues related to database truncation mentioned above:
# spec/spec_helper.rb
RSpec.configure do |config|
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, type: :feature) do
driver_shares_db_connection_with_specs = Capybara.current_driver == :rack_test
if !driver_shares_db_connection_with_specs
DatabaseCleaner.strategy = :truncation
end
end
config.before(:each, :truncation_mode) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
Changing Application Settings
Rails provides a number of settings that can be easily configured based on the environment, so we avoid undesired work in development or test environments, such as sending emails. For any mature Rails application, we’ll likely have our own custom settings layered on top of the Rails defaults.
There are many cases where we’ll still want to test the production settings in our test environments. For example, controller caching is disabled in the tests by default:
# config/initializers/test.rb
Rails.application.configure do
# ...
config.action_controller.perform_caching = false
end
For selected acceptance specs, we may still want to test behavior of caching at the view layer, say that users can see new info when a model attribute changes. We don’t need this caching behavior, so it may be useful to toggle specs on/off during the test run.
First Attempt
We could try to stub the setting in the context of a single spec run with the enabled state.
# spec/spec_helper.rb
RSpec.configure do |config|
config.before(:each, :caching) do
allow_any_instance_of(ActionController::Base).to receive(:perform_caching).and_return true
end
config.after(:each, :caching) do
Rails.cache.clear
end
end
This may require changing behavior of instances which is typically discouraged. We may also need to clean up other global states, like clearing the Rails cache after the test run.
A Better Attempt
Alternatively, we can set the actual values on while settings are derived. Here’s how it might look for enabling controller caching with an around
block:
# spec/spec_helper.rb
RSpec.configure do |config|
config.around(:each, :caching) do |example|
caching = ActionController::Base.perform_caching
ActionController::Base.perform_caching = example.metadata[:caching]
example.run
Rails.cache.clear
ActionController::Base.perform_caching = caching
end
end
The around
block takes the RSpec example object as an argument. When running specs, the given block is triggered when :caching
is detected as a key in an exampleβs metadata. The example object provides a number of methods for test introspection, allowing us to make changes before and after calling run to execute the spec.
As a result, we now have a simple and explicit mechanism for introducing caching to individual specs and suites:
# spec/features/homepage_spec.rb
describe "visit the homepage", :caching do
it "expires cache" do
# test cached stuff
end
end
The main concern with this approach is that modifying a global state can affect other tests unintentionally β a big no-no.
To avoid this, we need to reset the original value when the example completes.
Here, we are storing the previously set value of ActionContoller::Base.perform_caching
, setting it for the local suite, and resetting it back to the original value after it completes.
This technique may come into play when integrating with certain gems like PaperTrail, which may generate expensive logic or queries that aren’t needed in most cases. PaperTrail even provides a helper to take advantage of RSpec. It may be worth considering whether to provide an interface to toggle behavior and RSpec helpers next time we write gem metadata to toggle behavior in specs.
Filtering Specs
One useful technique while developing is running a selected set of specs. We may be editing acceptance specs, model validations, and other disparate tests while test driving a feature from outside in.
Manual tagging
Adding arbitrary metadata like :focus
to a set of specs is one way of approaching this.
# spec/models/user_spec.rb
it "validates a user", :focus do
# unit test
end
# spec/features/sign_up_spec.rb
it "displays error message", :focus do
# acceptance spec
end
We can now filter our test run to a subset at the command line as follows:
$ rspec --tag focus
We can also add some global configuration so this will be the default behavior when using :focus
specs, as long as we don’t make the mistake of unintentionally filtering on the build server.
RSpec.configure do |config|
# enable auto-focus only when running locally
config.filter_run_including focus: ENV['CI_SERVER_SETTING'].blank?
config.run_all_when_everything_filtered = true
end
Alternatively, avoid running broken or flaky specs when tagged accordingly:
it "test that fails intermittently", :flaky do
# probably a JavaScript test
end
Using either a command line option
$ rspec --tag ~flaky
or a configuration option, we can filter out specs we wish to ignore.
RSpec.configure do |c|
c.filter_run_excluding flaky: true
end
Auto Tagging
A less-known feature of RSpec 3 is an API for telling RSpec to derive additional metadata automatically based on the other metadata.
For example, each spec example has metadata that includes its file path. This, along with the RSpec::Core::Configuration#define_derived_metadata
method, allows us to alter spec behavior based on the spec directories, for example.
Why is this useful and how do we use it? Glad you asked.
Let’s say we want to isolate model specs that require database truncation since they are more like functional specs than unit specs. We may set up our spec directory as follows:
spec/
truncation/
example1_spec.rb
example2_spec.rb
...
transaction/
example1_spec.rb
example2_spec.rb
...
Instead of manually tagging each file with our :truncation_mode
metadata we used earlier to toggle DatabaseCleaner’s truncation strategy, we can configure all the specs in spec/truncation
as follows:
# spec/spec_helper.rb
RSpec.configure do |config|
config.define_derived_metadata(file_path: %r{spec/truncation}) do |metadata|
metadata[:truncation_mode] = true
end
# rest of DatabaseCleaner config below
end
Now, all specs in the directory will run with the :truncation_mode
metadata, and the database strategy will be set to :truncation
as long as it is declared ahead of the additional DatabaseCleaner configuration we referenced earlier.
Note that this is the same method used in rspec-rails
to add custom behavior to specs in specific directories, e.g. spec/controllers',
spec/requests, etc.
Using and Abusing
While using RSpec metadata can be a powerful technique for altering test behavior and application settings in specs, it can also be taken too far.
As @avdgaag notes in his blog post on the topic, make sure to distinguish between how a spec is run from what the spec should test. For example, we probably shouldn’t use metadata to create records specific to certain tests, or authenticate users for a given context.
One rule of thumb for adding metadata is to decide whether it would be generally useful in any Rails app (good), or if it’s specific to the business logic of your current application (bad). The latter is best set up more explicitly within or alongside your tests.
Before considering a new metadata tag, I ask the rubber duck “Could I extract this configuration into a gem?” To answer yes, the behavior would have to be non-specific to my application. If so, the behavior might be useful as metadata.
While metadata can nicely separate the boilerplate required to set up and tear down test behavior, it also adds a layer of indirection that can cause readability issues when stretched too far. Understand that there is a big increase in mental overhead to permuting test behavior with each new tag option, and consider the tradeoffs with the rest of the team.
Use it wisely!
If you have any questions or comments about this article, feel free to leave them in the comment section below.
P.S. Would you like to learn how to build sustainable Rails apps and ship more often? We’ve recently published an ebook covering just that β “Rails Testing Handbook”. Learn more and download a free copy.