​When the stars are aligned, use Agile

The high-risk and high-reward Agile approach for systems development enabled many organisations to respond quickly to changing management strategies and yielded significant productivity benefits, according to a 2015 survey.

More than half (53 per cent) of the respondents to the survey – conducted by US-based VersionOne – had a positive outcome from Agile as opposed to 47 per cent that had lower than expected outcomes.

However, the same survey found not everyone has been so successful, as lack of experience in using the Agile approach, and organisation resistance to change, have frustrated almost the same number of companies.

Once IT and business management have decided that Agile is the right approach they must do the following:

• Champion and defend its use
• Actively track progress and allocate extra resources to the project if justified
• Provide a safe environment in which a retrospective review can be conducted
• Widely disseminate the lessons learned from the review, including strategies that succeeded and failed, without attributing blame.

Until the early 2000s, most organisations followed the waterfall approach to systems development as it is based on engineering principles with clearly defined processes, stage gate reviews, and comprehensive documentation.

But with the availability of the Internet, and the power of the informed consumer armed with smart phones and devices, management has been forced to respond with a sense of urgency – hence the attraction of the Agile methodology as outlined in the diagram below.

The Agile approach initially attracted management’s attention as it bypassed the slow but sure waterfall approach and quickly reduced the development backlog.

However, a recent survey of developers found that using a mix of Agile and waterfall yielded higher quality and changeable code and a more secure environment than Agile alone.

The contrasting findings in the above surveys probably indicate that success was achieved when the attributes in the table below were present and unlikely to occur when they were absent.

Aligning the stars

When the following attributes are present, use Agile:

Attribute	Why important
Right sized and skilled team available	Ideal Agile team size is seven plus or minus two for socialisation and effective contribution in scrums
Outcomes that can be achieved quickly, e.g. in eight weeks to enhance existing on line service or update pages of a web site	Outcomes must be clear so effort is rightly directed – if not, effort will be misdirected and the team frustrated
Scrum master able to act as thought leader	Role of scrum master is pivotal in direction setting and resolving ambiguities
Process for running daily scrums is well understood	Team members committed to making Agile work, honest in presenting achievements and obstacles
Insightful and decisive product owner	Important the product owner is a visionary, can make decisions without delaying progress and confirm completion of the task (done)
Disciplined management of sprints, ideally of two work weeks duration	Ensure work completed is logged as ‘done’ and update the product backlog, as in diagram below, at the end of a sprint
Task estimating is achieved by informed consensus of team	Task estimates must be reliable for workload planning and variance reporting, i.e. actual to estimated work hours
Architects are active members of the team	Involve architects (Solution, Infrastructure and Security) in systems design and performance management reviews to minimise risks
Rigorous testing is carried out before work is Done (completed)	Imperative quality or error free code is confirmed and not compromised to meet schedules
Credible retrospective review process conducted after implementation	The reviews provide a valuable learning tool and provide feedback on how to replicate success and minimise risks

The list above is indicative and not exhaustive. There are many variations to Agile such as XP (extreme programming) and KANBAN. Organisations typically cherry-pick aspects of Agile that are feasible within their culture or business model and their staff’s capabilities.

Hybrid projects typically have known requirements, use a daily scrum, and employ rigorous testing before deployment. Some approaches include small batches (KANBAN), short releases (XP), and minimal work in progress.

Management’s dilemma

If Agile is not the right fit, use the following guidelines to determine the best approach:

Attributes/environment	Agile/hybrid	Use waterfall
Functional enhancements to a business critical application	Use if >15 but <40 work months, requirements are known, and the team is in the same location	If >40 work months and rigorous testing is essential
Enterprise-wide software implementation and major systems integration effort	Unwise as stakeholder systems assimilation is problematic and structured integration testing is required	So stakeholders own the solution. The structured approach facilitates systems assimilation
Development teams in multiple locations	Unwise as team logistics could be an impediment	Preferred approach as it will reduce risks and facilitate stakeholder assimilation

Caveats

• Avoid acquiring IT infrastructure required (on premises or external) until systems design is complete and transactions volumes known so right computing resources are obtained

• One criticism of Agile is it could lead to a lack of robustness (partially tested code), security exposures, and lack of changeability (ease of maintenance). This criticism emerged in a small and selective survey in which 75 per cent of respondents claimed Agile only lead to lower quality code.

To minimise the risks involve security professionals in the design phase, call out recalcitrant developers with sparse documentation, and insist rigorous testing be done before signifying completion or done. When operational failures occur include the reasons in the retrospective review, as in the diagram below.

• If planning to engage a service provider using Agile on a time and materials basis, keep sprints to a minimum to minimise costs. Ideally prepare workload estimates in advance to keep the providers honest

• Only allocate staff to Agile projects that are self-starters, thrive in a fluid environment, and are team players. Staff who lack these attributes are likely to be frustrated and demotivate others

Next steps to consider

• If planning to start an Agile project conduct a review to ensure the essential attributes in the table above are present and heed the warnings under caveats
• If an Agile project is in progress, and may fail due to exceeding resource estimates, stop development, assess risks, and determine the changes required including for instance adopting a hybrid approach with daily scrums and a revised backlog.

Alan Hansell is an IBRS advisor who focuses on IT and business management. Alan has extensive experience in IT management, consulting and advising senior managers in matters related to IT investment. He was a director in Gartner's Executive program and adviser to over 50 CIOs and business managers and before joining Gartner a consultant with DMR Group. He also worked as an IS professional, manager and industry consultant for IBM for nearly 30 years.