Nowadays, Agile methodologies are applied and well-known all over the software development industry. But are they actually implemented as they should?
According to Software Architect and author Allen Holub, while Agile is the way to go, it does not mix well with other practices just as widespread in the industry—chiefly, estimates and project-based development.
Edited transcription
Despite starting as a hardware engineer building robots, Allen Holub built his career as a software architect. Over the years, Allen has been a prolific author and an Agile consultant dedicated to helping companies understand what is Agile, and, more importantly, what is not, or as he puts it, to expose fake Agile.
What is #NoEstimates about?
In software development, estimates detail the resources (mainly time and money) projects will take to be complete before starting. Estimates are fundamental for project-based software development: From a business perspective, organizations use estimates to define the budget and timeframe for the work to be done; for developer teams, it can help them to break down the work into followable steps and lay out the complexity of their projects.
While estimates are common in software development, Allen challenges the whole concept radically, arguing that both estimates and project-based development are incompatible with software development best practices and, more specifically, Agile methodologies. More precisely, he understands that tying the budget into the project collides with Agile methodologies, making it impossible for “real” Agile to be put into practice in such an environment. In choosing between estimates and Agile, Allen chooses the latter, regarded as fundamental for efficiently building software products. To him, software development can succeed without estimates: “the more Agile you are, the less you need them“.
Allen traces back estimates to the fear of an uncertain outcome. “The whole idea of projects and the whole idea of having estimates at all is because the person who’s spending the money is afraid of losing it”, he says.
While Allen recognizes this is a valid reason for fear, he understands that Agile proposes a way of working that diminishes uncertainty. First and foremost, trust between management and developers needs to be established for overcoming this fear. According to Allen, traditional teams develop trust in two ways:
- Hitting their estimates. Management will trust teams that hit their estimates. However, to hit their estimates, teams are prone to pad them out. Besides, the team will avoid delivering early, since it will establish a precedent for future estimates. In avoiding delivering early, teams will also pad out their work time.
- Delivering consistently and predictably. The idea is that teams who rightfully follow a project schedule are trustworthy. However, according to Allen, the problem with this approach is that “if you’re not delivering frequently you get back into that realm of fear.” Management will have a hard time trusting when weeks or months pass without new deliveries, even if they are scheduled for later: “The longer time elapses, the more afraid you are and the lower trust levels are”.
Hence, to Allen, the best way to build trust is through delivering frequently and showing improvements over previous deliveries: “In that kind of environment, people stop asking for estimates because they don’t need them anymore and the fear goes away because you actually are delivering, so you don’t have to be worried about being in a situation where delivery isn’t going to happen.”
In this regard, ideally, “anything that we can do to make that feedback loop smaller is a good thing”. Allen encourages adopting methodologies that provide constant feedback from the customer, as in the case of extreme programming, in which there is an onsite customer role, and scrum, where there is the scrum product owner.
To administer funding without estimates, Allen says that the focus should be on funding teams’ time to work. After determining how much time the company can pay, he proposes answering what is valuable enough for teams to be working on. To start moving forward, Allen emphasizes having a defined strategic direction that should be open to feedback from the market: “The idea here is to figure out the direction “as” we go along and we’re willing to invest this much money because we expect this return.” Allen says that keeping an eye on how the market reacts to the product is another form of estimation, but not an estimation on a specification, but rather a dynamic investment strategy. This business model allows obtaining feedback from exposing a product to the market, learning if it is worth investing in it, and deciding if it’s valuable to invest in future products or not.
This is where management comes into the equation. Allen then prefers to talk about leadership rather than management: “in an agile world the teams manage themselves, so there’s no need for managers because what you need is leadership.” To Allen, to have an actual Agile company, management needs to know to take distance from the work of developers and limit their own role to:
- Establish the culture in the organization
- Setting a corporate strategy to follow
- Communicate the culture and strategy with the whole organization
Products over projects
Similarly to estimates, Allen finds that project-based development defies Agile methodologies. Allen affirms that the more Agile companies are, the less they rely on projects: “when you have less project focus you can focus on the product as a whole.” He understands that projects are prone to be judged according to whether or not predefined achievements have been met and not so much on what teams could have found about the product market or achieved beyond the initial expectations.
The fact that projects focus on achieving an end state contradicts the Agile mindset in which developers, when starting, have no way of knowing for certain what their product will look like. Agile methodologies limit the movement in preset strategic directions, and as the team delivers and receives feedback, they modify their product.
Allen also points out that, since projects are temporary, once a project is over, companies, especially large corporations, are prone to tear teams apart and put members into new teams for future projects. He believes that when the teams are torn apart, all the experience and chemistry between members go to waste. Moreover, the project product will miss the supervision of the very people who build it.
Entrepreneurship vs large companies
Over the decades, Allen has witnessed several changes in the software industry. In his experience, the most prominent —and negative— has been large corporations taking over the development of software, sizing small companies and SaaS tools in the process. According to Allen, most large corporations are not Agile and are to blame for the misconceptions about Agile and scrum.
On the other hand, Allen says that the business success of large corporations has led developers to seek the prestige of working for them. However, he argues that they are not good places to work: “Amazon, in particular, has a reputation for being something of a sweatshop, and, of course, Facebook, now Meta, is a company that is dedicated to the destruction of democracy.”
At the same time, the supremacy of large companies has resulted in their mentality becoming dominant in the industry. According to Allen, the very first professional experiences in software development are prone to mark developers’ patterns of thinking. Hence, they will prefer their ways of working over others: “If the first experiences you have working professionally as a programmer or for an Agile company, you will not want to do anything else; if you start working in a large, Agile-hostile corporate environment, you will also start thinking that that’s the only way that you can work.” In this regard, Allen says that management is not so much to blame for this lack of Agile but for developers who opt to work for corporations.
For this very reason, Allen recommends developers who have a hard time finding work on Agile companies to start their own projects: “You just find some piece of software that you would like to have that doesn’t exist and build it, and see if anybody wants to buy it.” Despite the financial risk entrepreneurship comes with, he finds it to be as risky as working for a big company: “Companies have no allegiance to you as an employee.” Moreover, Allen trusts that “companies that don’t have any agility are going to be driven out of business by companies that do.”
When it comes to small companies looking to grow, Allen recommends to “spin-off a subsidy that will be dedicated to a specific product and then leave them alone.” This will allow companies to preserve Agile methodologies and “accommodate to market changes quickly.”
The bottom line
You can follow and reach out to Allen on Twitter. He has a mailing list on his website you can subscribe to and receive updates on his public classes and more. Allen is currently working on a book, #No, centered on his ideas of no estimates, no projects, and fake Agile and how to recover from it. After ending this book, he plans to write #Yes, focused on effective software development and based on his list of Heuristics for Effective Software Development Organizations.
