Using Agile in less-than-perfect situations since Y2K
This is the second part of a long post on iterative and incremental development. In Part 1, definitions were provided to distinguish both practices from the general term “iterative development”. This part describes some pitfalls of doing one without the other and how each adds value to the other.
The pitfalls of only iterating without incrementing are simple to summarize: nothing new is added. That is, if iterating is the practice of improving what already exists, then it provides no facility to add functionality to a system. While — in theory — the product and/or process would eventually be as near to perfect as humans can make it, the overall value would not increase.
Though it seems contrived, this happens in real projects. For example, assume that a new project starts its first iteration with requirements that are not properly thought out. The product that is created will not make the customers happy. They will want changes: improvements. They iterate on the requirements themselves, resulting in iteration on the product. The project team then discovers that the new implementation could support the changed requirements better with a change in architecture. They convince the customer to allow them to make the improvements. Meanwhile, the customer has spent so much time thinking about this one requirements set that they start to “need” more bells and whistles. Soon, updates to the requirements are created, and the cycle begins again.
The pitfalls of only incrementing without iterating are a little more complex. If incrementing only adds to what exists, then it is not spending effort to improve what is already there. At an extreme, this would be like sealing each iteration result in a shared library file (DLL, jar, etc.) and requiring each subsequent iteration to only use those pre-compiled artifacts. In a more realistic example, this is simply not allowing time for refactoring. Another version is skipping the retrospective or not implementing any of the improvements suggested by the retrospective. Both result in creating or failing to remove hurdles that slow or block progress.
Incrementing without iterating is more common in my experience. It is somewhat easier to see the lack of value increasing when nothing new is being created. It is more difficult to tell when the rate of value increase is stymied by technical, project, or other debt.
If you spend a portion of each period of work improving what has been done before, you can remove those hurdles that prevent progress. By making sure that a portion of each period of work is spent on adding new features or practices, we reduce or prevent unnecessary gold-plating and particularly wasteful forms of analysis paralysis. In short, each practice fills gaps left by the other.
At some point (1994 according to Alastair Cockburn’s article), the project management and process community made the interesting decision to combine these to in the single term “iterative”.
Was it, as Alastair says, because they were just happy to be “repeating the cycle”? My hope is that it is more because they saw the need for both to work together. The term may have been a poor choice, but combining them is a great choice.
There are few people who argue against iterative development. However, it seems that while most of us consider iterative and incremental development to be part and parcel of iterations, not everyone seems to do both — at least not consciously and effectively. In a pair of posts, I will explain why both have more value together than either does separately. This post will build the ground-work, describing what each of these practices is and their separate value. Part 2 will describe how they become more valuable when used together and the pitfalls of doing one without the other.
Iterative development is a means of scheduling time to improve the process and product. Think about retrospectives. An amount of time is spent working on the product. A discussion of strengths and needed improvements is held. Then, an amount of time is spent reworking the product just built and the practices in use. The point is that you spend time working on the same pieces each work period (iteration).
This iteration on the same thing allows considered improvement. It is similar to refactoring. This allows for paying off of technical debt, enhanced security and performance, and improvements in understanding, testing, and implementation. It increases overall quality and efficiency of the system under development and the practices in use.
Incremental development is a means of growing a large product through delivery of a series of smaller pieces. Think about use cases or user stories or features. The overall product is described as a set of discrete pieces, each having value on their own (see INVEST). The pieces are related enough that when put together, they form a larger, cohesive system. In incremental development time is spent working on new pieces each work period (iteration).
This iteration on small pieces allows for each new increment to be considered for deployment to production. A release plan may set goals for timing or scoping of releases, but the actual ability to demonstrate the working portion may lead to a decision that a set of functionality that is delivered earlier may have enough value to warrant an early release. Delivery of properly prioritized increments also helps to avoid getting to the end of a schedule without something valuable to deploy. The most important (and risky) parts of the system are guaranteed to be there early.
While writing this post, I noticed that Alistair Cockburn wrote something similar in August of last year. His ending statement says in part that iterative development improves the product, while incremental improves the process. While Alistair is an authority and my experience and understanding is consistent with everything else he writes on that page, I disagree with this summation. It is more appropriate to say instead that iterative improves both product and process, while incremental builds upon both product and process.