CIO Upfront: Independent Assurance on ICT projects: A legal perspective
- 25 February, 2015 06:00
The rate of ICT project failure around the world remains high so anything that tackles common problem areas such as an ill-conceived program (for example, one that is too big or overly complex and therefore more likely to fail) or scope creep (a project killer if not picked up and addressed at an early stage) should be encouraged.
By way of background, the government is seeking to strengthen the existing assurance regime by asking the Government Chief Information Officer (GCIO) to assume responsibility for: “giving Ministers system-wide assurance that ICT risks are being identified and well managed by agencies and across government as a whole, and that the benefits of ICT investments are being delivered.”
This mandate includes the formation of independent assurance panels as summarised by the Department of Internal Affairs in a recent update.
“The Government Chief Information Officer (GCIO), as the functional leader for Government ICT, is mandated to establish an independent panel of Independent Quality Assurance (IQA) and Technical Quality Assurance (TQA) providers. The objective of the GCIO Assurance Services Panel is to improve the quality, consistency and independence of Assurance Services. This will provide greater confidence to Ministers and the public that investments are well managed and will deliver the expected benefits.”
The rate of ICT project failure around the world remains high so anything that tackles common problem areas such as an ill-conceived program or scope creep should be encouraged
This is intended to form part of and compliment the government’s existing approach to assurance generally which includes processes such as Major Projects Monitoring and Gateway reviews.
No guarantees
The Cabinet paper on this subject (June 2013) was quick to point out that: “Improving system-wide ICT assurance is intended to provide stakeholders with confidence that ICT risks and processes within the State Services are identified and effectively managed. While no assurance model can guarantee there will never be security or privacy breaches or service delivery failures, it can ensure risks are identified and managed [my emphasis added].”
I agree with this sentiment. As we know, even the best laid plans can come unstuck at some point during the planning, procurement and delivery of ICT projects. These projects carry inherent risk and it is therefore impossible to “assure” them to the point that a success rate equal to 100 per cent can be guaranteed.
I nonetheless expect the government’s increased focus on assurance to result in material improvements provided of course that:
• the assurance work is properly performed; and
• any recommendations made are in fact adopted.
It would also be useful to get clarity around what “success” means on an ICT project in order that, over time, the government’s initiatives in the assurance area can be properly measured. For example does success mean on time, on budget with all the required features, functions and benefits (a high standard, particularly on larger, more complex programs) or something short of that?
Challenges
What then are the likely challenges (or pitfalls to avoid) when it comes to implementing the government’s assurance model?
For present purposes I have selected the following: ensuring true independence; working with incomplete information; failure to follow recommendations from assurance reviewers; and risks associated with “smoking gun” documents.
Ensuring true independence
Ensuring true independence (of review teams) may be a challenge, particularly in a relatively small market like ours. In some situations there may be a case for involving reviewers from outside New Zealand in order to avoid any actual or perceived conflicts of interest.
Independence is fundamental to the effectiveness or otherwise of assurance work. Without independence, any assurance work is likely to be undermined or of limited value.
In this context, the “principles” published by the UK’s Major Projects Authority (set up in 2011 to undertake assurance work) provide useful guidance on what independence means:
“The Major Projects Authority works with HM Treasury and other government departments to provide independent assurance on major projects.
The ... Review team must be independent [my emphasis added] of the programme/project, its management and associated support activities and is responsible for the content of the final report.”
The objectives behind this principle are described as follows:
• “objective assessment of projects by teams with no association with [the] project or its line management
• avoids [leadership] influencing review outcomes and team conflicts of interest
• encourages open and candid reporting
• ownership of [the] report rests with [the Review Team] until [the] final version is delivered to [leadership].”
But what does this principle mean in practice?
Page Break
In short, it means that the quality and usefulness of any review work undertaken may be undermined where reviewers have conflicts of interest, are not objective and do not report on an open and candid basis.
In other words, it is unrealistic (and contrary to the independence principle) to ask people directly engaged on (or connected with) a project to effectively “mark their own work” or the work of people with whom they have an association.
Similarly, open and candid reporting is required in order to expose any areas that need to be addressed. Senior officials need to know the facts in order that an informed decision on how to proceed can be taken.
Incomplete information
In order to be effective, review teams will be heavily reliant on documents and other information provided to them by interviewees.
If relevant material is omitted then assurance processes may be undermined: reviewers will be working from incomplete information and problem areas may remain undetected. Issues or problems that remain hidden are only like to get worse over time.
It is therefore vital that reviews are set up in a way that promotes full co-operation by interviewees. In my experience people tend to be less co-operative if they are suspicious of the inquiry or do not receive adequate comfort about its purpose and how information they provide will be used.
Problem areas need to be identified and flushed out in order that they can be addressed before it is too late.
A few simple pointers should pay dividends in this regard:
• Explain the context and process to the interviewee (including confidentiality);
• Explain that the process is intended to be positive and constructive;
• Highlight the downside of problem areas remaining undetected (and therefore the importance of being open and honest); and
• Allow adequate time for interviewees to prepare.
Recommendations by assurance reviewers that are ignored by agencies or departments
And what about situations where the review team does its job but senior public sector officials refuse to adopt the recommendations made? This has been a problem area in the UK and was picked up by a (UK) Public Accounts Select Committee last year who found that:
"... the MPA [the UK body with responsibility for assurance work, as described above] only has informal influence over departments. It supports the Treasury in approval and funding decisions but there is no obligation on the Treasury to follow its recommendations. It has no powers if a department decides to proceed with a project against MPA advice.” "It needs to have stronger, more formal mechanisms for driving change, and there should be transparency where ministers or officials have rejected its recommendations.” As a result, the Select Committee recommended that:
• the Major Projects Authority be given more power to drive change; and
• there be greater transparency if ministers or other officials do not follow recommendations from assurance reviewers.
These recommendations appear sensible and may be worth considering here in New Zealand, even at this early stage.
Typical problem areas for reviewers to target may include the following (this is a selection only – not an exhaustive list):
Pre-contract
• Is the department or agency buying too much (test “needs” versus “wants”)?
• Is the project too big? If so, can it be broken down into smaller parts?
• Is there too much complexity in current business processes and, if so, are adequate plans in place to simply these?
• Are users actively engaged in setting requirements?
• Is the project framed as a business endeavour rather than an ICT project?
• Is the scope clear?
• Are the timescales realistic?
• Have any “agreement to agree” scenarios been minimised?
• Is the project adequately resourced (capability and quantity)?
• Are contingency plans in place?
Delivery
• Is staff continuity good?
• Are contractual obligations being performed (by all sides)?
• Have any variations (for example, to scope, timetable and associated commercials) been properly documented?
• Are “spot-checks” required to check for scope creep and/or delay?
• Does the project need to be re-set?
• Is a testing plan in place and being followed (check for any corner cutting)?
• Does internal reporting reflect the true position on project status?
• Is escalation necessary to address any outstanding issues? Have breach notices been issued where required?
The exercise should, in my view, be largely intuitive and based on experience particularly when interviewing project participants. Too much process (for example, long lists of questions or areas to cover) may run the risk of distracting participants from the key issues and root causes involved.
Next: Smoking guns
Page Break
Smoking guns
This is another area to keep an eye on.
Legal questions could arise over the status of reports and other documents produced during the review process, particularly where they contain potentially damaging material: for example, admissions of fault (by interviewees) for problems experienced.
Are interview notes and reports (including drafts) protected by legal professional privilege or any other type of confidentiality?
Could the material be the subject of a successful Official Information Act (OIA) request?
What about production in any future legal proceedings between the agency or department and third parties?
These are not straight-forward questions to answer and review teams should seek the appropriate guidance, preferably before work begins (and before documents are generated).
At the same time, great care should be taken not to undermine the fundamental purpose of the exercise which is to report on an “open and candid” basis (to borrow and repeat the wording from the Major Projects Authority principles).
Problem areas need to be identified and flushed out in order that they can be addressed before it is too late.
Are interview notes and reports (including drafts) protected by legal professional privilege or any other type of confidentiality?
Australian support for assurance regimes
In Australia, support for the use of assurance regimes can be found in comments by The Hon Richard Chesterman QC in his 2013 report into the (disastrous) Queensland Health payroll system project: “[My recommendation is that] The Queensland Government apply an appropriate structure to oversee large ICT projects...
Whatever form of project management is adopted, it ought to have the following attributes:
... 2. they [i.e., the body and individuals given this responsibility] be vested with the authority to probe and report...
3. they have the ability to report to very senior public officials...and make recommendations, especially if deficiencies in project or contract management are detected
It should essentially have an assurance function [my emphasis added].”
See also the Australian government’s approach to assurance reviews and risk assessments generally.
I mention this for completeness because it fits with and supports the renewed focus on assurance here in New Zealand.
In conclusion, the New Zealand government’s renewed focus on assurance is a welcome move.
Independent assurance reviews are a handy risk management tool and should have a positive effect in this regard.
Independence is crucial and reports should be objective, open and candid.
The UK experience is instructive - it makes sense that those charged with responsibility for assurance reviews should have the power to drive change and that senior government officials should be asked to explain any decisions not to follow advice or recommendations made.
In setting up reviews, those involved should keep in mind that documents produced may not be protected by confidentiality at the end of the day – but, at the same time, ensure that reports remain open and candid in order that the purpose of the review exercise is not undermined.
Success on major projects is more likely where problems or issues are forced out and resolved at an early stage. History tells us that, in many cases, ICT failures could have been avoided (or at least mitigated) if steps had been taken at an early stage to face up to problems and, where required, re-set or re-baseline the program.
Michael Bywell is a consultant at law firm Minter Ellison Rudd Watts (michael.bywell@minterellison.co.nz)
Send comments and contributions to CIO Upfront to divina_paredes@idg.co.nz
Follow Divina Paredes on Twitter: @divinap
Follow CIO New Zealand on Twitter:@cio_nz
Sign up for CIO newsletters for regular updates on CIO news, views and events.
