Behavior-driven development (BDD) is about minimizing the feedback loop. It is a logical step forward in the evolution of the practice of software development. This article explains the concept and its origins.
If you are a software developer or an engineering manager, you are probably familiar with the Waterfall model, recognizable by the following diagram:
What was later named “Waterfall” was first formally described by Winston Royce in his 1970 paper “Managing the development of large software systems”. I recommend reading the whole paper to understand the idea. Most people learn about it secondhand and assume that this process was presented as the ultimate solution at the time. However, Royce recognized that having a testing phase at the end of the development process is a major problem:
I believe in this concept, but the implementation described above is risky and invites failure. […] The testing phase which occurs at the end of the development cycle is the first event for which timing, storage, input/output transfers, etc., are experienced as distinguished from analyzed. […] The required design changes are likely to be so disruptive that the software requirements upon which the design is based and which provides the rationale for everything are violated. Either the requirements must be modified, or a substantial change in the design is required. In effect the development process has returned to the origin and one can expect up to a 100-percent overrun in schedule and/or costs.
This model is still used to develop software in many companies across the world, for various reasons. Waterfall implies flow, but in practice there are always feedback loops between phases. All major improvements to the model over time have been made by ways of minimizing the feedback loops and making them as predictable as possible. For example, if we write a program, we want to know how long it will take us to find out if it works. Or, on the other hand, if we design a part of the system, we want to learn if it is actually programmable and verifiable, and at what cost.
So, when we look at a feedback loop, we look for methods we can use to minimize it. At first, our goal is to remove obviously wasteful work. Later, we start realizing that we are able to optimize and do things faster and better than we could have ever imagined back when we were doing things the old way.
The First Optimization: Test-First Programming
The first optimization happened by looking at the Coding and Testing phases. In the traditional quality assurance-based (QA-based) development model, a programmer writes code and submits it in some way to the QA team. It takes a day, a few, or weeks to get a report if the code works, and if the rest of the program works as well. A lot of times there are bugs, so, although we thought the work was finished, we need to go back to programming and fix all issues.
To cut down the feedback loop, we start coding and verifying at the same time: we write some code, and then we write some tests for it. Tests produce an excellent side effect, the automated test suite, which we can run at any time to verify every part of the system for which we have written a test. We soon begin to wish to have a test suite that covers the entire system, so that we can work as safely as possible.
The feedback loop of coding followed by testing still takes some time, so we invert it: we start writing tests before writing a single line of code. The feedback loop becomes very small, and it doesn’t take long before we realize that we are writing only the code we need in order to pass the tests that we write.
This is called test-first programming. When we work test-first, we use the tests we write to help us “fill in” the implementation correctly. This reduces the number of bugs, increases programmer productivity, and positively affects the velocity of the whole team.
Once we have a continuous loop of testing and coding, we’re still doing all our program design up front. We’re using test-first programming to make sure that our code works, but there’s a feedback loop where we may find out (disturbingly late) that a design is difficult to test, impossible to code, performs badly or just doesn’t fit together with the rest of the system as we try to implement it.
To minimize this loop, we apply the same technique. We invert it by doing test-first programming before we start designing. Or rather, we do the Testing, Coding and Program Design all at the same time. A test influences the code, which in turn influences the design, which influences our next test.
Soon we notice that this cycle drives our design ideas in an organic way, and we implement only the parts of the design that we need, in a way which can easily evolve too. Design now includes a substantial refactoring step, which gives us confidence to under-design instead of over-engineer. That is, we end up with just enough design and appropriate code which meets our current requirements.
This is test-driven development (TDD). It combines test-first programming with design thinking by continuous application of refactoring principles and patterns. The positive side-effects are now amplified: we have not only reduced the number of bugs but we are also not writing any code that doesn’t help us implement a feature. This further increases a team’s productivity by helping it avoid design mistakes which are more costly to fix the later they are detected.
Making the Next Step with Behavior-driven Development
Now that we are designing, coding and testing in one loop, we look back at the Analysis. By analysis I assume “understanding what we need to build”. Again, we’re interested in optimizing the loop because it takes time. In practice it would involve preparing a list of a dozen or more features and passing it on to developers, who need to complete them all before moving forward. This way we often end up in a situation of implementing features that we don’t need. Sometimes we also discover new features we didn’t expect we would need, or discover something new about the features we know we need.
We can apply the same technique and bring Analysis in our loop. Now we test-drive a feature before we try to implement another. It is worth illustrating that the duration of such cycles for a developer is measured in hours, sometimes even minutes, not days or weeks.
After applying this technique consistently for a while, we notice that we tend to break down all features in the smallest units and consistently deliver them one by one. We improve our understanding of how features affect one another and find ourselves being able to respond to changes faster. We can identify and discard unwanted features quickly and give priority to important features.
By test-driving our analysis, we better understand the behavior of the system that we need to build and how to appropriately design and implement it. At the same time, all that we are doing from day one is producing a test suite, which keeps our entire system constantly verifiable.
This is called behavior-driven development. It saves time of both the stakeholders (business owners) and the development team. By asking questions early, developers help both themselves and the stakeholders gain a deep understanding of what it is that they are building. Stakeholders get results at a predictable pace and, since the features are worked on in small chunks, estimates can be done more accurately, and new features can be planned and prioritized accordingly.
Going Even Faster
If you take a look at the remaining phases listed in the Waterfall diagram, which all need to happen regardless of the methodology, you may wonder if the same feedback loop minimization can be applied to them as well. The answer is, of course, yes. However, such loops are of a scope that is broader than just design and development, and they involve people working across very different fields, which means that they are out of the scope of this article’s topic. However, I will mention them briefly.
Lean Startup would be the closest concept that brings together gathering requirements, feature development and marketing as a way to close the loop on learning what a startup needs to build. Of course, the process goes somewhat differently in enterprises, although they are learning to apply the lean startup principles in many projects as well.
Merging BDD with deployment and operations brings us to the broad concept of continuous delivery. The most important processes are continuous integration (CI) and continuous deployment, which you can easily configure for any project on Semaphore.
Behavior-driven development evolved from optimizing various phases in the software development process. By analyzing, testing, coding and designing our system in one short feedback loop, we are able to produce better software by avoiding mistakes and wasteful work.
It is a common misconception that TDD is about testing and that, since it has its origins in TDD, BDD is just another way of approaching software testing. This is not the case, although tests are a nice byproduct. It is a holistic approach to software development, derived from one simple idea: wanting to optimize the feedback loops in our work.
None of the steps required to practice BDD are required to make software in general. They also take time to learn and apply effectively. Still, their payoff of a sustainable process that enables us to continuously produce working software is well worth the investment.
P.S. Would you like to learn how to build sustainable Rails apps and ship more often? We’ve recently published an ebook covering just that — “Rails Testing Handbook”. Learn more and download a free copy.