There is an age-old struggle in software development between getting things done fast and getting things done right. One of my biggest concerns about our industry’s new love affair with Agile Process is that Agile rigs the rules of the game to always favor the “do things fast” side of the debate.
How does Agile rig the game?
First, Agile requires us to describe the entire project in terms of what the user can see — “user stories”. This leaves no (official) room to think about the project from a technical implementation perspective. It would be as if the construction of a house were defined entirely in terms of color choices and flooring options, with no consideration made for the strength of the foundation or the structure.
Second, Agile requires us to “commit” to completing a set of user stories each sprint, again with no official space provided to work out technical details that the user can’t see. This would be like asking your contractor to get those granite countertops installed by week 2 of the project, even though the plumbing and electrical aren’t hooked up yet.
By describing the entire project in user-visible chunks and rewarding the team for completing its commitment to code up a batch of them each sprint, we create huge incentives to “just move the cards”, rather than to build quality software. This process tells your developers that when faced with a decision to implement a card in a quick and dirty way that allows them to complete the sprint versus a better way that would push the card to the next sprint, they should always default to choosing the quick-and-dirty, finish-the-sprint method.
How do we level the playing field?
I should note that I am not in favor of always choosing the “right” way instead of the “fast” way; I simply think that the decision needs to be made intelligently and thoughtfully. It certainly shouldn’t be made for us by the artificial time pressure of a sprint deadline. Therefore our goal should be to level the playing field so that we can make an intelligent choice on each fast-versus-right question. However, because agile tips the scales towards “fast”, we need strategies to tip it back towards “right”:
- Pad your estimates. Every Agile developer learns this one pretty fast. Inflating the estimate for a user story when you know you need to do some related (or perhaps unrelated) technical work is the easiest path to working within the Agile process while still doing the technical work Agile doesn’t want to recognize.
- Include technical work in your sprint. While it is very good to write up user stories and understand exactly what the users want, that shouldn’t mean that we can’t write up technical “stories” that describe the “invisible” work that needs to be done to do the project “right”. By openly discussing technical issues and including them in your sprint planning process, the importance of these issues is automatically elevated.
- Be willing to miss a sprint. Sprint deadlines are arbitrary. Don’t let yourself feel bad about occasionally missing a sprint in an effort to complete work in a technically sound way. You are the developer, ultimately it is up to you to decide how the code should be written. Take advantage of this power.
- Communicate with your entire team. Constantly discuss the fast-versus-right tradeoff with the entire team, developers and non-developers alike. Explain the downsides of implementing something the hacky way. Explain that spending an entire sprint on implementing a single user story the “right” way will allow you to implement seven similar stories next sprint.
Agile creates rules that rig the game in favor of fast development. By understanding this, you gain the power to bend the rules back towards good development, ideally creating a level playing field where everyone on your team can have a real conversation about the tradeoffs of each fast-versus-right debate.
I think you’re confusing agile as a whole with certain implementations or narrowed versions of approaches that come under the agile umbrella. It is not the case that agile approaches leave no room for technical details, architecture or infrastructure concerns. Just because the focus is on going fast does not mean that quality or sustainability should be sacrificed – these are specifically mentioned in the agile software development principles.
Agile does not necessarily require the use of user stories, though these are a good lightweight alternative to fully detailed use cases. When features are described in user stories though the rationale for the feature and the interaction described are from the user’s point of view (because that is where value is derived) it does not mean that they should not include tasks and estimated weight for non-user-facing parts of the story. A user story describing how a user wants to click some button on screen and see a result should necessarily contain the effort required to create the architecture, infrastructure, operations and monitoring concerns that can be attributed to that functionality.
Agile does not require that teams “commit” to delivering a certain number of user stories within a “sprint”. What you are presumably talking about here is a use of the Scrum framework to organise the agile project. Nonetheless, when the team in Scrum “commit” to work within a sprint it does not mean that there is no space, time or consideration provided to technical prerequisites or to the underlying systems upon which the features delivered will operate. As I mentioned before a good definition of a user story will, when estimated, contain all of the tasks included between discussion of the feature and delivery into an operational environment.
It is the collective responsibility of the team and their facilitators/management to ensure that the work done isn’t just hacked together and moved from ready to done with no consideration of operations concerns. A development team that lacks discipline in this fashion may do so due to inexperience, lack of care, lack of oversight or a lack of understanding of the operational environment. This is not an agile problem – this is a lack of communication and cultural closeness.
You should not “level the playing field” – you should raise these concerns with the team and with management so that everyone can address them visibly. Taking an underhand approach is less sustainable and less effective compared to an honest and collaborative discussion.
– Do not pad your estimates. Agile methods recognise the need to address technical debt (see agile principle #9).
– On your remaining points you are correct. Non-feature technical work must still be included and addressed in the development of the system. Working towards deadlines may be important but ensuring the quality of the features and system is paramount to creating, evolving and operating software. Don’t compromise to finish something off early with poor quality. The point you make about communication is the most important – this will lead to the outcomes you desire, but attempting to ‘game’ the system will not. Being open and honest and visible about work required and the details beyond user-facing features is necessary to ensure success.
Nice post and good point about being willing to miss a sprint! Never missing a sprint would indicate to me that the team is under-committing or papering over deficiencies.
The other thing I think people often fail to consider is that “user story” does not have to mean “customer story”. A story that includes “as an engineer” is a perfectly valid story and should be given points and planned for accordingly.
An interesting perspective and arguably a very common problem.
For me, the concept of ‘definition of done’ is the mechanism for addressing those all important non-functional/technical requirements that always need to be considered. As Mat says, there is no reason not to have a standalone user story either – whilst the business may appear to focus on features, there is usually an implicit expectation that the core non-functional requirements will be met. What product owner (or equivalent) wants a new feature that brings the site/service/app down – sometimes we just need to remind ourselves that there is that implied expectation, which is why I think having a concrete definition of done is so useful.
Anwyay, thanks for the blog, there are very few that take a Microsoft devops angle.
James, Mat, & Steve — thanks for your comments. You have all made really good points.
Even though I do kind of enjoy trash talking about Agile, I don’t think that every Agile process automatically implies a preference for doing things fast versus doing them right. The point I am trying to make here is that it is all to easy to fall into the trap of de-prioritizing doing things right using some of the tenets and trappings of “Agile” process as a cover. It is up to us (it is our process after all) to ensure that we don’t fall into that trap and instead create a process that is small-a-agile — producing high quality software that delivers value and responds to change quickly. I propose a few strategies for avoiding the trap, and you guys have offered a few more excellent suggestions.
I think that all of the strategies we’ve discussed can be boiled down to a basic guiding principle — make sure that your process accounts for the technical work required to “do things right”. This accounting should be done as openly as possible (Steve, I agree that communicating effectively about this is a much better option than simply padding estimates). It can be done by having explicit “non user” stories (using Mat’s “as an engineer” template), by tasking out a story into technical and operational components (as Steve suggests), or (per James) by “defining done” to include some metric of technical quality.
In the end, no out-of-the-box process (Agile or otherwise) will solve your problems. You need people who are focused on doing their jobs well and are communicating effectively about the design, tasks and trade-offs.