8 Sep 2022 · Software Engineering

    Applying the Test Pyramid Concept to Ruby on Rails Apps

    14 min read
    Contents

    The Test Pyramid is an important concept when it comes to designing your test suite. However, Ruby on Rails and RSpec have their own terminology for test types. It can be tricky to know on which level of the pyramid your tests reside.

    This article will explain how the test types available on Rails and Rspec relate to the different levels of the pyramid. It will also show how this mapping will help you determine how many of each type to write.

    This article is structured into the following sections:

    1. Scope
    2. Introducing the Test Pyramid
    3. Ruby on Rails Test Types
    4. Planning for Ideal Test Coverage
    5. Conclusions
    6. Other Posts You Might Be Interested In

    1. Scope

    The practical advice in this article relates to a typical Rails app (a monolith). It does not cover a microservices scenario. We also assume that most view logic is processed in the backend, with small sprinkles of Javascript.

    2. Introducing the Test Pyramid

    The Test Pyramid is a way to classify the different kinds of tests used in software development. It divides the tests into three layers:

    The Test Pyramid

    Unit tests evaluate small pieces of the software in isolation. The test code calls the unit’s methods and evaluates if the actual output corresponds to the expected values. Unit tests are easy to develop and fast to execute. However, they are very sensitive to changes in the design. If you change the internal structure of your software, you’ll have to adjust the unit tests.

    On the middle level of the pyramid, there are integration tests. They test how different components work together. The test code calls a method of a higher-level component that interacts with other lower-level components to produce a result. They will take a little more time to develop when compared to unit tests but are less sensitive to changes in application design.

    The top level consists of end-to-end tests. They interact with the application’s UI as if a human was conducting manual tests. The test works by clicking buttons and links, entering values, and evaluating what the UI is showing. They require a lot of work to write and are slow to execute.

    3. Ruby on Rails Test Types

    When writing tests in Rails, you can choose from the standard Rails Test Toolkit – which is based on Minitest, or use an external test framework: RSpec being the most popular.

    Both frameworks define a set of test types to choose from:

    RSpec TerminologyRails (MiniTest) Terminology
    Model SpecsModel Tests
    Controller SpecsFunctional Tests
    Request SpecsIntegration Tests
    System Specs / Feature SpecsSystem Tests
    Helper SpecsHelper Tests
    View SpecsView Tests
    Routing SpecsRouting Tests
    Mailer SpecsMailer Tests
    Job SpecsJob Tests

    To make the terminology clear, in this article we will use the terms unit test, integration test and end-to-end test when referring to the layers of the pyramid and RSpec terminology (model specs, system specs, etc.) when referring to the test types defined within the framework.

    3.1. Rails Tests within the Test Pyramid

    The figure below positions the Rails test types within the testing pyramid:

    Rails Test Types and the Test Pyramid

    We will now see examples of each type, using a sample to-do application. The examples are written in RSpec, but the same ideas apply to Minitest as well.

    It is very important to note that the test types on RSpec relate to specific artifacts of the Rails Framework. With a large Rails application, it is quite common to have lots of classes representing different business objects and abstractions that do not exist in the framework. They should, of course, be tested. Such tests of POROS (Plain Old Ruby Objects) can be classified as unit tests or integration tests, depending on how they’re written.

    3.2. Model Specs

    Model specs are unit tests that test functionality from your Rails models.

    Our example to-do app has a ToDo model:

    class ToDo < ApplicationRecord
      validates :title, presence: true
    
      def overdue?
        due_date.present? && due_date < Date.today?
      end
    end

    We need to test the overdue? method. Here is how we could do it:

    RSpec.describe ToDo, type: :model do
      
      describe 'overdue?' do
        it 'past due date' do
          todo = ToDo.new(title: 'write blog post', due_date: Date.yesterday)
    
          expect(todo).to be_overdue
        end
    
        it 'due today' do
          todo = ToDo.new(title: 'write blog post', due_date: Date.today)
    
          expect(todo).to_not be_overdue
        end
    
        it 'future due date' do
          todo = ToDo.new(title: 'write blog post', due_date: Date.tomorrow)
    
          expect(todo).to_not be_overdue
        end
    
        it 'blank due date' do
          todo = ToDo.new(title: 'write blog post', due_date: nil)
    
          expect(todo).to_not be_overdue
        end
      end
    
    end

    3.3. Controller Specs

    There is usually no need to write this type of spec. The controller methods should be simple and are already covered by request specs. In fact, as of RSpec 3.5, both the Rails and RSpec teams discourage directly testing controllers in favor of request specs.

    3.4. Request Specs

    Request specs are a convenient way to test your entire application stack without the big overhead that comes with system specs. For this reason, we recommend that you write request specs for all controller actions.

    There should be at least one test case for each HTTP response. Here is an example of testing the ToDosController#create action:

    describe "POST /create" do
    
        context "with valid parameters" do
          it "creates a new ToDo" do
            expect {
              post to_dos_url, params: { to_do: valid_attributes }
              
            }.to change(ToDo, :count).by(1)
          end
    
          it "redirects to the created to_do" do
            post to_dos_url, params: { to_do: valid_attributes }
    
            expect(response).to redirect_to(to_do_url(ToDo.last))
          end
        end
    
        context "with invalid parameters" do
          it "does not create a new ToDo" do
            expect {
              post to_dos_url, params: { to_do: invalid_attributes }
    
            }.to change(ToDo, :count).by(0)
          end
    
          it "renders a response with http status 422" do
            post to_dos_url, params: { to_do: invalid_attributes }
    
            expect(response).to have_http_status(:unprocessable_entity)
          end
        end
      end

    3.5. Route Specs

    Route testing is typically done as a part of request specs. When done separately, however, it can be classified as a unit test. That said, unless you have very complex logic in your routing rules, there should be no need to write specific tests for your routes.

    3.6. View Specs

    You can use view specs to test the content of view templates without invoking a specific controller.

    For example, the welcome screen of our to-do app shows a customized greeting message. This is how we could check its content using view specs:

    RSpec.describe "home/index", type: :view do
      context 'with overdue tasks' do
        it 'shows welcome message with warning' do
          assign(:overdue_to_dos_count, 3)
    
          render
    
          expect(rendered).to include 'Get to work!'
        end
      end
    
      context 'without overdue tasks' do
        it 'shows welcome message with congrats' do
          assign(:overdue_to_dos_count, 0)
    
          render
    
          expect(rendered).to include 'Congratulations'
        end
      end
    end

    Note that there is no need to write view specs if you just want to make sure that your view code will cause no runtime errors, since request specs will already ensure that.

    3.7. Helper Specs

    Helper specs check the functionality of helper methods.

    The sample to-do app uses a helper method to display a status for each to-do on the list:

    module ToDosHelper
      def to_do_status(to_do)
        if to_do.due_date.past?
          tag.span 'overdue', class: 'overdue'
        elsif to_do.due_date.today?
          tag.span 'due today', class: 'today'
        else
          # don't display anything if the date is blank or in the future
        end
      end
    end

    The helper specs for this method check the output for each case:

    RSpec.describe ToDosHelper, type: :helper do
      context "blank due date" do
        it "returns nothing" do
          to_do = ToDo.new(due_date: nil)
    
          expect(helper.to_do_status(to_do)).to be_blank
        end
      end
    
      context "past due date" do
        it "returns 'overdue' label" do
          to_do = ToDo.new(due_date: Date.yesterday)
    
          expect(helper.to_do_status(to_do)).to include 'overdue'
        end
      end
    
      context "due today" do
        it "returns 'due today' label" do
          to_do = ToDo.new(due_date: Date.today)
    
          expect(helper.to_do_status(to_do)).to include 'due today'
        end
      end
    
      context "future due date" do
        it "returns 'due soon' label" do
          to_do = ToDo.new(due_date: Date.tomorrow)
    
          expect(helper.to_do_status(to_do)).to include 'due soon'
        end
      end
    end

    The use of helper methods together with helper specs allows you to test small pieces of a view in isolation.

    3.8. System Specs

    Standing at the top level of the pyramid, System Specs are End-to-End Tests that use a headless browser to simulate user interactions with the application and exercise the whole stack.

    Let’s write a test that simulates an end-user creating a to-do Item. Here is the interaction we want to test:

    • Start by accessing the to-do list
    • Click on “new to-do”
    • Fill in the form
    • Save the record
    • Verify that the created to-do appears on the todos/show page

    We use capybara commands to write the system spec:

    RSpec.describe "Managing To-dos", type: :system do
      before do
        driven_by(:selenium_chrome_headless)
      end
    
      it "Create and display a to-do" do
        visit "/to_dos"
    
        click_on "New to-do"
    
        expect(page).to have_text "New to-do"
    
        fill_in "Title", with: "do the thing"
        click_on "Save"
    
        expect(page).to have_text "To do was successfully created."
        expect(page).to have_text "do the thing"
      end
    end

    System specs take more time to write when compared to request specs, but they will give you a higher level of confidence. It is also important to note that there is a lot of overlap between the coverage given by both types. It is not necessary to have request specs for actions that are already covered by system specs. But it is up to you to decide on the balance between the two.

    3.9. Mailer Specs

    There are two aspects of testing your mailers. One is ensuring that the mailer is doing its job of setting the email headers and content, which is a unit test. The other is testing if the other entities are correctly using the mailers. This is an integration test.We’ll look again at the to-do app for an example of unit testing mailers. The ToDosMailer class has a method to send a daily message:

    class ToDosMailer < ApplicationMailer
    
      def daily_reminder
        @to_dos = ToDo.where('due_date <= ?', Date.today)
            
        mail to: 'user_email@somedomain.com',
             subject: 'To-dos daily reminder'
      end
    
    end

    The corresponding mailer view renders different messages depending on whether the list stored on the @to_dos instance variable is empty or not. For this reason, we need at least 2 test cases:

    RSpec.describe ToDosMailer, type: :mailer do
      
      describe "daily_reminder" do
        
        context "with 1 to-do with past due date" do
          it "renders the body" do
            ToDo.create(title: 'write blog post', due_date: Date.yesterday)
            
            mail = ToDosMailer.daily_reminder
    
            expect(mail.body).to include "You have 1 to-do"
          end
        end
    
        context "without any to-dos" do
          it "renders the body" do
            mail = ToDosMailer.daily_reminder
    
            expect(mail.body).to include "You have no to-dos"
          end
        end
    
      end
    
    end

    3.10. Job Specs

    Similarly to mailers, two aspects of jobs should be tested:

    • the behavior of the jobs themselves
    • ensuring that jobs get enqueued by other entities

    The first part is a simple unit test that requires no special tooling. For the second, Rails provides a special queue adapter and some custom assertions to help testing.

    Suppose that we wanted to add an action in the to-do app, where the user exports a to-do item to an external system. This could take a long time, so let’s use an asynchronous job.

    The controller action that calls the job is implemented like this:

    class ToDosController < ApplicationController
      before_action :set_to_do, only: %i[ show edit update destroy export ]
    
      (...)
    
      def export
        ExportToDoJob.perform_later(@to_do)
      end
    
      (...)
    end

    To write a request spec for it, we can use the custom assertions provided by RSpec:

    RSpec.describe "/to_dos", type: :request do
    
      (...)
    
      describe "POST /export" do
        it "enqueues a ExportToDoJob" do
          to_do = ToDo.create! valid_attributes
          
          post export_to_do_path(to_do)
          
          expect(ExportToDoJob).to have_been_enqueued
        end
      end
      
    end

    Running this test will not execute the job. It will just check if the job has been added to the queue.

    4. Planning for Ideal Test Coverage

    Now that we have seen some practical examples of applying the test types, let’s discuss what we mean by a good test coverage, and why we want to achieve it.

    One way to think of the scope that you have to cover when testing an application is in a 2-dimensional space.

    On one axis, there are the application layers. You need different types of specs for each one. Some layers can be tested in isolation, others can only be verified by integration and/or end-to-end tests. And even for the layers that can be tested in isolation, you still need to verify that the integration is working.

    On the other axis, you have the input space. The different features and inputs that your application has to handle. Even when you consider a single functionality, there can be different inputs and preconditions that will trigger separate paths in the application’s code. Testing many different combinations of inputs can only be done efficiently by unit tests. The figure below illustrates this idea:

    Dimensions of Test Coverage

    It should be clear from this diagram that a combination of different test types is necessary to achieve good coverage of the different layers and components of the application, and also of the problem space, i.e. the different features and inputs.

    4.1. A Cost-Benefit Analysis

    Even though we want to test things thoroughly, there are limits. Some decisions in testing end up in a cost-benefit analysis. For example:

    • Should I write lots of system specs, or can I rely mostly on request specs?
    • Should I write tests for model validations?

    What are the costs and benefits involved in the equation? Let’s start with the benefits. Even if the team does not practice TDD or BDD, many rewards can be obtained from a good test suite. You certainly want to:

    • Catch errors before they hit production
    • Increase developer efficiency
    • Be able to make changes without breaking things
    • Improve code quality

    However, those benefits will not come for free. There are some costs involved. The most relevant ones:

    • development effort: writing and maintaining tests represents work on the part of the developers.
    • execution time: the more tests you have, the longer it takes to run them (especially end-to-end tests).

    In the end, it is a cost-benefit analysis. But the math is far from simple. First, there are different measurement units. Some factors are in work hours, some are in execution time. Some variables are subjective, e.g. quality,  and occur in different timeframes. For example, the increased maintainability will pay off only in the long term.

    Intuitively, we can think of it like this: in a small test suite, as you add more tests, you are getting benefits that surpass the costs. Tests are helping developers structure the code, and spot potential bugs. They are saving more developer hours than they cost. But if you add too many, then they start to overlap, i.e. multiple tests target the same code. Changes in the code become more difficult to make. Developers start spending more hours fixing the tests than working on the code. So, if we were to depict this cost-benefit relationship in a (highly simplified) graph, it would look like this:

    Cost x Benefit according to test suite size

    The challenge here is to stay in the blue zone. The goal is to add enough tests, but not too many, with a smart combination of different test types.

    5. Conclusions

    In this article, we covered the available test types on Rails (minitest) and RSpec, and how they relate to the different levels of the classic test pyramid. We also discussed some strategies for effectively designing a test suite, illustrated with practical examples.

    The decision of which tests to write is influenced not only by technical aspects, but also by other organizational factors, such as team size, and how critical the application is. No answer will fit all scenarios, but by having a clear understanding of the pros and cons of each test type you can choose the testing strategy that works best for your situation.

    One thought on “Applying the Test Pyramid Concept to Ruby on Rails Apps

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    Avatar
    Writen by:
    I'm a full stack Ruby on Rails developer from Brazil. I love to code, learn about new technologies, and share my knowledge through blogging.
    Avatar
    Reviewed by:
    I picked up most of my soft/hardware troubleshooting skills in the US Army. A decade of Java development drove me to operations, scaling infrastructure to cope with the thundering herd. Engineering coach and CTO of Teleclinic.