BDD Testing a Restful Web Application in Python

Introduction

Behaviour-driven development allows you to describe how your application should behave, and drive the development of features by adding new tests and making them pass. By clearly describing how your application behaves in different scenarios, you can be confident that the product delivered at the end meets the requirements you set out to deliver. Following BDD lets you build up your application piece by piece, and also provides you with living documentation of your entire system, that is naturally maintained as you keep the tests passing.

By the end of this tutorial you should be able to:

• Create a simple REST application using the Flask framework
• Write behaviour tests (also known as acceptance tests) using the Lettuce library
• Explain the structure of the tests, in terms of the Given, When, Then, And syntax
• Execute and debug the tests

Prerequisites

Before you begin this tutorial, ensure the following are installed to your system:

• Python 2.7.x
• Lettuce
• Flask
• Nosetests
• A basic understanding of REST principles

Set Up Your Project Structure

In this tutorial, we will build up a simple RESTful application handling the storing and retrieval of user data. To start, create the following directory structure for the project on your filesystem, along with the corresponding empty files to be added to later:

 .
├── test
│     ├── features
│            ├── __init__.py
│            ├── steps.py
│            └── user.feature  
└── app
     ├── __init__.py
     ├── application.py
     └── views.py

The files can be described as follows:

• __init__.py: mark directory as a Python package.
• steps.py: The Python code which is executed by the .feature files.
• user.feature: The behaviour test which describes the functionality of the user endpoint in our application.
• application.py: The entry point where our Flask application is created and the server started.
• views.py: Code to handle the registration of views and defines the responses to various HTTP requests made on the view.

Create the Skeleton Flask Application

For the purposes of this tutorial, you will need to define a simple web application using the Flask framework, to which you will add features following the BDD approach outlined later in the tutorial. For now, let’s get an empty skeleton application running for you to add to. Open up the file application.py and add the following code:

from flask import Flask
app = Flask(__name__)

if __name__ == "__main__":
    app.run()

This code simply creates our Flask instance, and allows you to start the packaged development server Flask provides when you execute this Python file. Should you have everything installed correctly, open up a command prompt on your operating system and execute the following command from the root of the project:

python app/application.py

If you see the following output, then your Flask application is running correctly, and you can proceed with the tutorial:

$ python app/application.py
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Write Your First BDD Test

As we want to follow BDD, we will start by writing the test first which describes the initial functionality we want to develop in our application. Once the test is in place and failing, we will proceed to writing the code to make the test pass.

Write the Feature file

Edit user.feature and add the following code to the first line:

Feature: Handle storing, retrieving and deleting customer details

This first line is simply documentation for what functionality the set of scenarios in this file cover. Following this, let’s add your first scenario:

  Scenario: Retrieve a customers details

Again, this line is simply documentation on what functionality this specific scenario is testing. Now, let’s add the actual body of the scenario test:

  Scenario: Retrieve a customers details
      Given some users are in the system
      When I retrieve the customer 'david01'
      Then I should get a '200' response
      And the following user details are returned:
        | name       |
        | David Sale |

You will notice the test makes use of the standard set of keywords known as gherkin (e.g. Given, When, Then, And). The syntax provides structure to your test, and generally follows the following pattern:

• Given: the setup or initialisation of conditions for your test scenario. Here, you might prime some mocks to return a successful or error response for example. In the test above, you ensure some users are registered in the system so we can query it.
• When: the action under test, for example making a GET request to an endpoint on your application.
• Then: the assertions/expectations you wish to make in your test. For example, in the above scenario, you are expecting a 200 status code in the response from the web application.
• And: allows you to continue from the keyword above. If your previous statement began with When, and your next line begins with And, the And line will be treated as a When also.

One other important thing to note is the style of the test and how it reads. You want to make your scenarios as easy to read and reusable as possible, allowing anyone to understand what the test is doing, the functionality under test and how you expect it to behave. You should make a great effort to reuse your steps as much as possible, which keeps the amount of new code you need to write to a minimum, and keeps consistency high across your test suite. We will cover some techniques on reusable steps later in the tutorial, such as taking values as parameters in your steps.

With Lettuce installed to your system, you can now execute the user.feature file from the root directory of your project by executing the following command in your operating system’s command prompt:

lettuce test/features

You should see output that is similar to the following:

$ lettuce test/features/

Feature: Handle storing, retrieving and deleting customer details # test/features/user.feature:1

  Scenario: Retrieve a customers details                          # test/features/user.feature:3
    Given some users are in the system                            # test/features/user.feature:4
    When I retrieve the customer 'david01'                        # test/features/user.feature:5
    Then I should get a '200' response                            # test/features/user.feature:6
    And the following user details are returned:                  # test/features/user.feature:7
      | name       |
      | David Sale |

1 feature (0 passed)
1 scenario (0 passed)
4 steps (4 undefined, 0 passed)

You can implement step definitions for undefined steps with these snippets:

[ example snippets removed for readability ]

You will notice here that our tests have obviously not passed as we have not yet written any code to be executed by our feature file. The code to be executed is defined in what is known as steps. Indeed, the output from Lettuce is trying to be helpful and provide you with the outline for the steps above for you to fill in with the Python code to be executed. You should think of each line in the scenario as an instruction for Lettuce to execute, and the steps are what Lettuce will match with to execute the correct code.

Define Your Steps

Underneath the feature file are the steps, which are essentially just Python code and regular expressions to allow Lettuce to match each line in the feature file to its step which is to be executed. To begin with, open up the steps.py file and add the following imports from the Lettuce library:

from lettuce import step, world, before
from nose.tools import assert_equals

The key things to note here are the imports from Lettuce, which allow you to define the steps and store values to be used across each step in the world object (more to follow). Also, the imports from the nose tests library, which allow nicer assertions to be made in your tests.

Now you will add a @before.all step (known in Lettuce as a hook, which, as the name suggests, will execute some code before each scenario. You will use this code block to create an instance of Flask’s inbuilt test client, which will allow you to make requests to your application as if you were a real client. Add the following code to the steps.py file now (don’t forget to add the import statement towards the top of your file):

from app.application import app


@before.all
def before_all():
    world.app = app.test_client()

With the test client in place, let’s now define the first step from our scenario, which is the line Given some users are in the system:

from app.views import USERS


@step(u'Given some users are in the system')
def given_some_users_are_in_the_system(step):
    USERS.update({'david01': {'name': 'David Sale'}})

The step adds some test data to the in memory dictionary, which, for the purposes of this tutorial application, acts like our database in a real system. You will notice the step is importing some code from our application, which you will need to add now. USERS is an in-memory data store, which, for the purposes of this tutorial, takes the place of the database which would likely be used in a real application. Let’s add the USERS code to the views.py file now:

USERS = {}

With this in place, you can now define the next step, which will make the call to our application to retrieve a user’s details and store the response in the world object provided by Lettuce. This object allows us to save variables, which we can then access across different steps, which otherwise would not really be possible, or would lead to messy code. Add the following code to steps.py:

@step(u'When I retrieve the customer \'(.*)\'')
def when_i_retrieve_the_customer_group1(step, username):
    world.response = world.app.get('/user/{}'.format(username))

In this step definition, notice how a capture group is used in the regular expression allowing us to pass in variables to the step. This allows for the reuse of steps talked about earlier in the tutorial and gives you a great deal of power and flexibility in your behaviour tests. When you provide a capture group in the regular expression, Lettuce will automatically pass it through to the method as an argument, which you can see in this step is named username. You can of course have many variables in your step definition as required.

Next, you will add your first assertion step, which will check the status code of the response from your application. Add this code to your steps.py file:

@step(u'Then I should get a \'(.*)\' response')
def then_i_should_get_a_group1_response_group2(step, expected_status_code):
    assert_equals(world.response.status_code, int(expected_status_code))

Here you make use of the assertion imported from the nosetests library assert_equals, which takes two arguments and checks if they are equal to each other. In this step, you again make use of a capture group to put the expected status code in a variable. In this case, the variable should be an integer, so we convert the type before making the comparison to the status code returned by your application.

Finally, you need a step to check the data returned from your application was as expected. This step definition is also a good example of how Lettuce supports the passing in of a table of data to a step definition, which in this case is ideal, as the data may grow quite large and the table helps the readability of what is expected. Add the final step to the steps.py file:

@step(u'And the following user details are returned:')
def and_the_following_user_details(step):
    assert_equals(step.hashes, [json.loads(world.response.data)])

In this step you can see that when you pass in a data table, it can be accessed from the step object under the name hashes. This is essentially a list of dictionaries for each row of the table you passed in. In our application, it will return a JSON string which is just the dictionary of the key name to the user’s name. Therefore, the assertion just loads the string returned form our application into a Python dictionary, and then we wrap it in a list so that it is equal to our expectation.

Executing the Scenario

With all your steps in place now, describing the expected functionality of your application, you can now execute the test and see that it fails. As before, execute the following command in a command prompt of your choice:

lettuce test/features

As expected the tests should fail with the following output:

$ lettuce test/features/

Feature: Handle storing, retrieving and deleting customer details # test/features/user.feature:1

  Scenario: Retrieve a customers details                          # test/features/user.feature:3
    Given some users are in the system                            # test/features/steps.py:17
    When I retrieve the customer 'david01'                        # test/features/steps.py:22
    Then I should get a '200' response                            # test/features/steps.py:27
    Traceback (most recent call last):
        [ SNIPPET REMOVED FOR READABILITY ]
        raise self.failureException(msg)
    AssertionError: 404 != 200
    And the following user details are returned:                  # test/features/steps.py:32
      | name       |
      | David Sale |

1 feature (0 passed)
1 scenario (0 passed)
4 steps (1 failed, 1 skipped, 2 passed)

List of failed scenarios:
  Scenario: Retrieve a customers details                          # test/features/user.feature:3

As you can see, our application is currently returning a 404 Not Found response, as you have not yet defined the URL /user/<username> that the test is trying to access. You can go ahead and add the code now to get the test passing, and deliver the requirement you have outlined in your behaviour test. Add the following code to views.py:

GET = 'GET'


@app.route("/user/<username>", methods=[GET])
def access_users(username):
    if request.method == GET:
        user_details = USERS.get(username)
        if user_details:
            return jsonify(user_details)
        else:
            return Response(status=404)

The code first registers the new URL within your Flask application of /user/<username> (the angled brackets indicate to Flask to capture anything after the slash into a variable named username). You then define the method that handles requests to that URL and state that only GET requests can be made to this URL. You then check that the request received is indeed a GET and, if it is, try to look up the details of the username provided from the USERS data store. If the user’s details are found, you return a 200 response, and the user’s details as a JSON response, otherwise a 404 Not Found response is returned.

If you execute your tests from the command line once again, you will see they are now all passing:

$ lettuce test/features/

Feature: Handle storing, retrieving and deleting customer details # test/features/user.feature:1

  Scenario: Retrieve a customers details                          # test/features/user.feature:3
    Given some users are in the system                            # test/features/steps.py:17
    When I retrieve the customer 'david01'                        # test/features/steps.py:22
    Then I should get a '200' response                            # test/features/steps.py:27
    And the following user details are returned:                  # test/features/steps.py:32
      | name       |
      | David Sale |

1 feature (1 passed)
1 scenario (1 passed)
4 steps (4 passed)

You have now delivered the functionality described in your behaviour test, and can move onto writing the next scenario and making that pass. Clearly, this process is an iterative cycle, which you can follow daily under your application is delivered in its entirety.

Additional Tasks

If you enjoyed following this tutorial, why not extend the code you have now by behaviour-driven development testing the following additional requirements:

• Support POST operations to add a new user’s details to the USERS data store.
• Support PUT operations to update a user’s details from the USERS data store.
• Support DELETE operations to remove a user’s details from the USERS data store.

You should be able to reuse or tweak the currently defined steps to test the above functionality with minimal changes.

Conclusion

Behaviour-Driven Development is an excellent process to follow, whether you are a solo developer working on a small project, or a developer working on a large enterprise application. The process ensures your code meets the requirements you set out up front, providing a formal pause for thought before you begin developing the features you set out to deliver. BDD has the added benefit of providing “living” documentation for your code that is, by its very nature, kept up to date as you maintain the tests and deliver new functionality.

By following this tutorial, you have hopefully picked up the core skills required to write behaviour tests of this style, execute the tests and deliver the code required to make them pass.