Agile 2008 – ideas and inspiration

Agile 2008 is an international conference in Toronto on Agile software development. It’s held from the 4th to the 8th of August. It’s gaining in popularity with 1600 participants from all over the world. Of course most attendants are from the US and Canada.
I attended this conference and gave a presentation together with Olav Maassen. I want to share some of the inspiring ideas and ‘vibes’ that I picked up at the conference.

On to something

Being amongst Agile people for a whole week gave me a very rich and inspiring feel for what’s going on in the software industry. The Agile alliance really shines this year because the conference hosted many different people and ideas, some of which are promoting Agile while others are pointing out shortcomings of Agile and some sessions even had nothing to do with Agile but were still filled with great ideas. I think I’ve picked up a few inspirational ideas that I want to explore further.
The thing that struck me most is the ‘vibe’ of where software development will see big improvements in the next years. Coincidentally my own presentation actually had similar points to the ones I picked up in other presentations. I was definitely on to something.
Apart from a few particular sessions that I’ll describe in later blogs, there are two main themes that I picked up at this conference that might not be obvious. One theme I like to call Agile+ and the other is summarized best as Iteration-1

Agile+

Agile+ for me is everything that I’ve heard at the conference that tries to improve Agile development beyond the current principles and practices. A few ideas were refinements of Agile practices while others extend to software development adding to the Agile foundation.
The principles of the theory of contraints will give an Agile team a new way of looking at their process and possible improvements. One shortcoming of Scrum and XP is that there are no limitations in for the team to increase their work-in-process. Using the Lean Kanban technique to push the cycle time down and revealing bottlenecks makes the self-organising aspect of Agile a much more disciplined and fruitful affair. One obvious practice that was mentioned is to always fix bugs immediately, don’t let the tester report a bug and then wait 2 days to fix it, but leave the feature in ‘testing’ until all bugs are actually fixed keeping the WIP low. This fits into a larger theme of applying Lean and TOC to software development to extend Agile with more discipline and insights. The Kanban system is a very concrete implementation of this. David Anderson gave a convincing argument for Kanban. As an early adopter I would like to see some whitepapers and handles to introduce Kanban in Agile projects. The most striking differences to classic Agile is the abandonment of iterations and planning sessions.

Iteration -1

Scott Ambler made a somewhat cynical observation that a lot of talks on Agile2008 are about what we ought to be doing in Agile projects (such as automated acceptance testing, pair programming etc.) but that actual experience show that some practices are actually not very popular although they are immensely popular as a topic of discussion (such as automated acceptance testing). Agile+ will bring more research and real stories on what’s really going on in projects so we can discuss reality and not an idealized version of reality. Now that Agile is entering the mainstream we’ll need more facts and figures to increase adoption than the anecdotal and inspiring stories we’ve used in the past.
The Iteration-1 came up a few times, even at other conferences. Scott Ambler mentioned it explicitly, but David Thomas, Allen Cooper and Jeff Patton also have some notion of it. To it’s core is the ‘could-be-controversial’ idea to not start developing software from the very first iteration. People like Scott are protesting the notion that your very first iteration of the project should start with coding (as XP and Scrum seem to promote). This might be a situation where Jeff Sutherland would say “I’m not sure on which Agile planet they are…” but I have experienced the pressure on Agile projects to fire up the IDE and start coding from the start. Now several people are proclaiming to take certain steps or phases before you start cranking out ‘potentially shippable products’. Now what you should do exactly varies from modelling to design, architecture, interaction design or building throw-away software.
Almost everyone agrees that you cannot build the right solution before you’ve seen the whole problem or scope of the project. This intrinsic problem might lead you to spend very little time on design or architecture because you don’t know what the best solution will be until you’re done. This is where Agile might overshoot in it’s focus on software and feedback. You can go around this smarter. You can use more than just ‘working software’ to find out what your customer really needs. Allen Cooper even says that even live software won’t give you the right feedback because people tend to have a bad idea what the really need even when they see it. So more time needs to be spend on discovering what features your users really need. So Iteration-1 starts before any coding starts. Scrum doesn’t describe what you do before the coding starts where except for ‘provide vision and backlog with features’. That gap needs to be filled.
I see one fundamental dilema with the Iteration-1, which is the Lean principles of minimising waste and work in progress. Design and architecture can be viewed as unvalidated artifacts that increase the cycle time of your features. On the other hand you can see a project as a discrete event that has a relative fixed set of problems to fix. Lean (and in some ways Scrum) seem to be geared more toward a continuous flow of features. This is great, but you’ll have to make sure that those features fit a coherent set, fit a coherent architecture and a coherent feature set. That’s where you’ll need a view of the whole system again. So if your starting a whole new project in a new context, you can’t start churning out features from the start, you’ll need some Iteration-1 (or -2, -3).

Conscious Development

I think a lot more attention will be spent on conscious preparation and applying requirements engineering, interaction design and architecting before the coding phase. This is of course a little like waterfall and a lot like the RUP iterations. The fact that people call it Iteration-1 shows they’re still a bit embarrassed trying to incorporate it with the Agile principles. Hopefully we’ll get over it and give it a proper name and give it a proper place in software development.