Agile is, in part, a reaction to the rigid process models of the past. It is no surprise that many throw formality out as well. However, without a degree of formality, we lose repeatability. This has happened in countless efforts to implement Agile methods. The team, seeing a chance to break from the rigidity, pushes off any attempts at formality as well. As the project progresses, cracks begin to form. There is no formal test plan, so high-level bugs creep in. That is, while the expressed stories may work as requested, certain aspects are simply not covered. Because there was no formal effort to design the user interface flow, usability of the product starts to plummet as the complexity of the combined requirements weighs upon each successive screen.

These, and other aspects, are things we as software development professionals have learned to do reasonably well in the last 30-plus years. Ignoring them hurts our chances of success. Are these things non-Agile or anti-Agile?

No they are not. To completely ignore formality would be to throw even Agile practices out the window. Software development requires people to coordinate actions. The amount of formality to do so varies based upon the team and its goal. However, some degree of formality can be an aid when more than one person is involved.

It is all in how we apply formality. Things such as test plans, information architecture and user interface flow, and domain modeling applied in an “all up front” fashion would be difficult to reconcile. However, none of these things has to be rigid. They can adapt just as easily as we can refactor our code. If the plans are designed with this in mind, small adaptations are easy while larger ones remain possible.

It is worth repeating that not every effort requires the same degree or type of formality. If a project is just starting or has only minimal and very simple user interactions, perhaps a comprehensive outline of user interface flow really isn’t necessary. However, we must train ourselves to recognize when it becomes necessary. As soon as anyone in a retrospective or user demo says, “I can’t find a screen where I can do X,” it might be time to examine the flow. In fact, it may be worth formalizing a list of “smells” that point to the need for adding some of these practices. Somewhat similar to those in Agile Adoption Patterns. What do you think?

The Warriors could optimize like crazy for the press, but they’d be training and working and trading and drafting to be, at best, a pretty good team.

This is why those that preach from the gospel of Lean try to impress upon us that local optimization is an insidious evil. It often looks good from a certain viewpoint, but it can hide other improvements that would provide a more holistic benefit. It can even prevent you from reaching your true goal.

Those of you who run in Agile circles should recognize those Author’s names. Esther wrote another Pragmatic book, Behind Closed Doors, with Johanna Rothman. I’ve read a number of her blog postings and articles over the years and generally like her style. She is also active at the Agile Conference, running the “Open Jam” stage/sessions. Diana was a director of the Agile Aliance from 2004-2006 and speaks at number of conferences.

The book follows a common pattern: a topic introduction, a set of “recipes”, and some wrap-up. In this case, the introduction covers the first three chapters. The first describes what a retrospective is, providing a framework for retrospectives by breaking them into five parts:

1. Set the Stage
2. Gather Data
3. Generate Insights
4. Decide What to Do
5. Close the Retrospective

The second chapter describes the importance of fitting the retrospective to the organization, schedule, and other factors. The advice here is generally good. Using the same activities in every situation is at best not going to be as successful as using activities properly tailored to the context.

Chapter 3 gives some advice on how to effectively lead a retrospective. I found this reasonably informative, but some of the specifically recommended dialog felt a little like “pop psychology.” For example, “I’m hearing labels and ‘you’ language.” However, that is not to say that it isn’t good advice, or that I haven’t used something very like it before. And, of course, having lead a few retrospectives, some of this may have been more obvious to me than it would to someone without that experience.

The majority of the book’s content is in Chapters 4 through 8. These work much like a prix fixe menu for your retrospective. That is, they provide a set of activities from which to choose in building a retrospective with the parts described in the first chapter. Each chapter focuses on activities for one of those parts. If you have ever been in a retrospective, you will likely have seen at least a few of these. You may also have seen some that are not mentioned. What makes this valuable is not that it is unique or complete. It serves as a reference when trying to build a retrospective. I can easily see a team turning to this each iteration, especially when their last retrospective may have seemed stale or less productive.

Chapter 9 outlines some of the differences between retrospectives at iteration end compared to project or release retrospectives. These can be different in multiple ways (length of history being discussed, number of people involved) and the book provides some good insight here.

Chapter 10 is short, but provides some good advice and tips for how to get a team (and to a lesser extent an organization) to follow through on the decisions made as a result of the retrospective. After all, retrospection without adaptation is worse than wasteful.

Overall, I like the book. That said, if you are about to lead a retrospective for the first time, this book will not take the place of having been in a retrospective or having a good coach. This book can serve as good preparatory material to help build your confidence and give you options for when things take unexpected turns.

I would recommend this book more strongly to those involved in or leading retrospectives and finding them to be getting stale or losing effectiveness. The lists of activities provided in the middle chapters might just help.

The pitfalls of only iterating without incrementing are simple to summarize: nothing new is added. That is, if iterating is the practice of improving what already exists, then it provides no facility to add functionality to a system. While — in theory — the product and/or process would eventually be as near to perfect as humans can make it, the overall value would not increase.

Though it seems contrived, this happens in real projects. For example, assume that a new project starts its first iteration with requirements that are not properly thought out. The product that is created will not make the customers happy. They will want changes: improvements. They iterate on the requirements themselves, resulting in iteration on the product. The project team then discovers that the new implementation could support the changed requirements better with a change in architecture. They convince the customer to allow them to make the improvements. Meanwhile, the customer has spent so much time thinking about this one requirements set that they start to “need” more bells and whistles. Soon, updates to the requirements are created, and the cycle begins again.

The pitfalls of only incrementing without iterating are a little more complex. If incrementing only adds to what exists, then it is not spending effort to improve what is already there. At an extreme, this would be like sealing each iteration result in a shared library file (DLL, jar, etc.) and requiring each subsequent iteration to only use those pre-compiled artifacts. In a more realistic example, this is simply not allowing time for refactoring. Another version is skipping the retrospective or not implementing any of the improvements suggested by the retrospective. Both result in creating or failing to remove hurdles that slow or block progress.

Incrementing without iterating is more common in my experience. It is somewhat easier to see the lack of value increasing when nothing new is being created. It is more difficult to tell when the rate of value increase is stymied by technical, project, or other debt.

If you spend a portion of each period of work improving what has been done before, you can remove those hurdles that prevent progress. By making sure that a portion of each period of work is spent on adding new features or practices, we reduce or prevent unnecessary gold-plating and particularly wasteful forms of analysis paralysis. In short, each practice fills gaps left by the other.

At some point (1994 according to Alastair Cockburn’s article), the project management and process community made the interesting decision to combine these to in the single term “iterative”.
Was it, as Alastair says, because they were just happy to be “repeating the cycle”? My hope is that it is more because they saw the need for both to work together. The term may have been a poor choice, but combining them is a great choice.

Iterative development is a means of scheduling time to improve the process and product. Think about retrospectives. An amount of time is spent working on the product. A discussion of strengths and needed improvements is held. Then, an amount of time is spent reworking the product just built and the practices in use. The point is that you spend time working on the same pieces each work period (iteration).

This iteration on the same thing allows considered improvement. It is similar to refactoring. This allows for paying off of technical debt, enhanced security and performance, and improvements in understanding, testing, and implementation. It increases overall quality and efficiency of the system under development and the practices in use.

Incremental development is a means of growing a large product through delivery of a series of smaller pieces. Think about use cases or user stories or features. The overall product is described as a set of discrete pieces, each having value on their own (see INVEST). The pieces are related enough that when put together, they form a larger, cohesive system. In incremental development time is spent working on new pieces each work period (iteration).

This iteration on small pieces allows for each new increment to be considered for deployment to production. A release plan may set goals for timing or scoping of releases, but the actual ability to demonstrate the working portion may lead to a decision that a set of functionality that is delivered earlier may have enough value to warrant an early release. Delivery of properly prioritized increments also helps to avoid getting to the end of a schedule without something valuable to deploy. The most important (and risky) parts of the system are guaranteed to be there early.

While writing this post, I noticed that Alistair Cockburn wrote something similar in August of last year. His ending statement says in part that iterative development improves the product, while incremental improves the process. While Alistair is an authority and my experience and understanding is consistent with everything else he writes on that page, I disagree with this summation. It is more appropriate to say instead that iterative improves both product and process, while incremental builds upon both product and process.