Our Blog

where we write about the things we love



The ins and outs of agile methodology

Over the past year, we have started to see a growing interest in implementing projects in an agile way by adopting agile methodology techniques. The agile approach suits a range of projects and engagements and the benefits to our clients can be measured in the successful outcomes we are seeing, the satisfaction with the end result and ultimately the reduction of risk and cost. This is not a new approach to delivering software and does not point to the demise of the traditional waterfall approach – both have their place.

Looking back over the last 30-40 years we have seen significant advances in the technologies and techniques supporting the software development process. However software engineering is still relatively immature and it is difficult to estimate accurately from the beginning. Compare the technology industry to the construction industry and I think you will agree we are still a long way behind.

The IT industry is littered with high profile failures which drive a considerable (and often excessive) focus on reducing the risk of project failure and cost blowouts. To minimise risk, project teams often conduct detailed requirements analysis - hence the adoption of the traditional waterfall approach and projects based on fixed scope, fixed features, fixed planning and fixed price. One of the main problems with this approach is that to apply waterfall techniques effectively to large and complex projects, the person hours and effort (not to mention the highly valuable and limited access to domain expertise) required to document every requirement up front makes the process time consuming and prone to deterioration of quality over time. Add to that the longer the requirements analysis effort takes the higher the probability that the underlying systems and processes will have changed and you see the quandary we find ourselves in IT so often. That is: how can we demonstrate and execute a project under a high degree of project control without spending the large up front effort associated with capturing all the requirements. The Standish Group Chaos report further emphasises this point: "72% of projects fail to deliver what was originally specified in the agreed time and budget." Because it is documented on paper doesn't guarantee project success.

So why do we attempt to document all the requirements up front? There are many reasons why this is tempting, some of which include:

  • Business cases need to be presented and cost is a major factor of this - how can we be confident on the total project cost if the low level detail is not known?
  • How do you compare vendors on one of the primary factors - price, if the detail is not known to price against?
  • Accountability - by requiring a fixed price commitment by the vendor,  the perception is lower project risk.  How can the vendor commit to a price if the detailed requirements are not known?
  • Fully specified and planned projects give the project team and management a greater feeling of confidence.

Is trying to understand everything up front the ideal approach for enterprise scale complex projects? I don't think so. This is where the agile approach shows its strengths. You still need to know the high-level requirements, the project scope and priorities, you still need budgets and planning (in fact agile processes require more planning and control) and you still need the core project teams and capabilities. A significant point of difference is that you don't try to gather all the detailed requirements up front before implementation. Instead you gather requirements only when you need them - and for an agile approach that is only for the next iteration. Agile projects deliver demonstrable working software with each iteration cycle, not just the one delivery as with traditional waterfall. An iteration cycle may only be four to six weeks in duration, but in that timeframe you fully analyse, develop and test (and potentially release) a working piece of software. This is where I believe the key benefits come through: as the iterations are small in duration, the project team can regularly assess real progress, re-evaluate priorities and features and change deliverables if required.

Another important attribute of the agile approach is to minimise change during an iteration cycle. As the iterations are short in duration and changes are evaluated at the start of each iteration, the potential negative effect of change management is minimised. As we know, complex projects will always change as the project progresses - business priorities change, key users will change their mind, new information becomes available and new technology choices may develop. The negativity surrounding traditional waterfall change management severely impacts the project progress and quickly invalidates the up front analysis and planning. With an agile approach you may not (and probably will not) end up with a solution that is exactly what you thought you needed at the start of the project; it is highly likely that you will end up with a solution that fits your needs.

OK, so if agile is so good, then why don't we all do it this way for all projects? I believe four primary factors prevent the agile approach being adopted:

  1. For many reasons the project itself does not suit an agile approach e.g. size, complexity, etc.
  2. The culture of the organisation doesn't lend itself to a project being executed successfully in an agile way.
  3. The agile approach relies on trust. You have to trust the team delivering the solution. The entire project team (including stakeholders) need to be honest and open and communication has to be highly effective and visible. If this is not possible, the agile will process will struggle to realise the benefits.
  4. Agile projects need to be managed well. If you do not have the project management competencies, agile projects will struggle.

As you can tell, I'm an advocate of developing software in an agile way - especially for complex long running engagements. The positive outcomes we are seeing support this approach and the positive feedback from our clients is also very encouraging. Due to a number of factors, we still conduct a significant number of successful projects following a traditional waterfall pattern; it's a matter of choosing the most appropriate approach for the situation.

Posted by: Tim Mole | 01 October 2007

Tags: Agile, Project Delivery

Top Rated Posts

Blog archive

Stay up to date with all insights from the Intergen blog