How to deliver IT projects on time and on budget

Information technology projects easily go awry because of their complexity
and the different groups of people involved in their design, testing and
implementation.

Fundamentally, the goals of these groups of people are in conflict. Project
managers want to go-live on schedule at all costs, while top management want to
stay inside budget. Business users are too busy in their day jobs to devote
quality time to system design and user testing, yet will complain the loudest
when the new system does not meet their needs. Meanwhile, outside stakeholders,
such as customers and the audit committee do not necessarily understand what the
fuss is about.

Being independent, internal audit has a central role to play in ensuring each
group avoids taking risky shortcuts to achieve their own objectives at the cost
of overall project success.

We have been involved in several IT implementations and this article
summarises many of the lessons learned during that time. In the most recent
example, the company had many legacy systems dating back around 20 years. Some
were bespoke applications; some were supported externally with little in-house
expertise or documentation; others were not compatible with the latest network
operating system. Data was either buried in systems with poor reporting tools
available for its extraction, or it resided on spreadsheets, in manual records
or peoples’ heads.

When the company decided to replace the old regime with new financial and
operational systems, it was an enormous undertaking and internal audit was
drafted in to help guarantee successful go-live.

We have identified systematic problems with IT projects and have turned these
into lessons learned for future projects.

1. Project scope

Project goals were outlined in project charters of varying quality. They were
often vague in respect of project deliverables, the impact of change on existing
procedures, predicted system usage and the quantifiable benefits of the new
system.

Although business management signed off acceptance of the charters, their
involvement in drafting them was suboptimal due to their day-to-day pressures in
running the legacy systems.

The resulting imprecise and narrowly-defined charters led to dissatisfaction
with the end product by the very people who failed to define what they wanted at
the outset. In the worst case, when we stripped out the gobbledygook, the
charter merely promised an ‘operable system’.

Lessons:

– The business must be involved at project initiation in defining project
objectives and scope.
– Deliverables should be clearly specified in the charter.
– There must be a distinct task before project closure to match the charter with
actual deliverables.

2. Business model

The new applications either overlaid existing ways of working, or ushered in
completely new functionality. Either way, users were forced to change the way
they worked. What the businesses discussed in concept, but never fully executed
in practice, was how the manual and IT processes would interact to provide
seamless data flows end-to-end.

Issues such as local procedures around a system, data primacy rules,
interfaces and other data transfer mechanisms, cross-system reporting and common
account structures were left out of the pre-implementation design as being ‘too
hard’, although in the post-implementation world they became even harder to
implement as the account structures had solidified. Project managers typically
used the excuse that these issues were “not go-live critical”. Don’t believe it!

Lessons:
– Detailed definitions of ‘to be’ business processes should be drafted,
which flow directly from the scope defined in the charter.
– Before acceptance of new processes, ensure there is proof of concept through
pilot testing.
– Ensure business sign-off for acceptance and ownership of new processes.
– Draft clear documentation of new business processes and their impact on
existing processes.

3. Benefits realisation

The benefits of the new systems were not always thoroughly articulated in the
project charters. Often, they were not aligned with deliverables, nor measurable
(for example, some stated: “greater ease of use of the system,” which is neither
articulate nor measurable).

Some projects identified greater bankable benefits than others because their
definition of a benefit was broader. Because of project time constraints, no
benchmarks were taken of the operating costs of the legacy systems before they
were decommissioned, nor of the new systems when they first went live.

Hence, time series comparison of how costs had reduced as a result of any
single implementation was not possible. Overall, it was difficult to match
actual cash outlays for implementations against the bankable benefits supposedly
accruing.

Lessons:
– Identify benefits at the business case stage of a proposed system –
link these to the project deliverables defined in the charter.
– Benchmark existing running costs (in legacy systems) to compare with on-going
benefits in the new production environment.
– Categorise benefits into ‘bankable’ and ‘non-bankable’ and define what is a
valid benefit.

4. Project management

Projects were supported by project managers and project plans. The plans
varied in standards of detail and comprehensiveness: the better the plan, the
better the project execution. To illustrate, individual tasks were specified
(for example, ‘execute test scripts – five days, or by dd/mm/yy’), but the
resources needed to do it were not always specific.

In one case, there was no global view of resource commitment and some staff
were over-committed. When staff could not achieve everything at once, it led to
a setback in milestone achievement or led to personal angst.

The need for cross-system liaison was not properly anticipated in individual
project plans. Eventually, insufficient time was allocated to interface testing
and end-to-end data flows, which led to a lot of manual intervention when
systems went live.

Lessons:
– Build resource allocation into project plans and get management to
commit those resources.
– Build cross-system forums.

5. Stakeholder involvement

Due to business-as-usual pressures, there was variable engagement between the
business users and the project teams.

Though project plans included business activities, their realisation was
patchy. There was inadequate business commitment of resources, particularly in
user acceptance testing (the ultimate proof of functional fit) and indifference
in making decisions before project deadlines.

Directors found it particularly difficult to devote regular quality time to
project board meetings. This risked a degree of cynicism among end-users before
go-live as preparation for change was not taken as seriously as it should have
been.

External consultants are typically used extensively on projects, but they are
expensive and often their loyalty lies outside the business. In one case, a
project manager left with insufficient notice two weeks before the project went
live. With their single-minded focus on go-live, project managers may take
decisions about system deliverables or project resources without business
ratification, especially if the business takes a hands-off approach to the IT
project.

Lessons:
– Enforce involvement by business managers in project implementations.
Ensure promises of loaned staff are honoured.
– Enforce attendance at project board meetings and report non-attendance.
– Ensure sign-off of key project milestones.
– Strengthen the business integration tasks in project plans – such as
communications, user training, documentation and change management.
– Use a project authority matrix to identify a process/system owner, budget
holder, design authority, end-client and project manager.
– Use terms of reference for project consultants to define their work and
authority to commit the company.
– Contractually oblige consultants to see projects through to completion and
knowledge transfer to the business.

6. Project budgets

Project budgets risk being inconsistent in their detail. Hardware costs,
inflation and accommodation charges were areas where different projects took
different approaches. A ‘requiring project pays’ policy could be applied.
Because many project requirements are ad hoc and made at the last minute, this
charging system may not always be applied.

Lessons:
– Standardise the basis of budget calculations across projects.
– Ensure budget headings match the key project processes.
– Enforce a robust and consistent ‘requiring project pays’ policy.

7. Go-live acceptance criteria

Deliverables were reviewed by the project boards before go-live. However, not
every deliverable carried an associated acceptance criterion. Criteria were not
always adhered to. Product deliverables were dropped from scope when it looked
like they could not be achieved or were rationalised as non-critical for go
live. In such scope changes, there needs to be business sign-off for the
decision, impact analysis and contingency arrangements.

Having an issues log is one thing. As one of the go-live acceptance criteria,
it needs to be cleared down so that all substantial issues are known to have
been addressed.

Where project deliverables had acceptance criteria, these could have been
more methodically analysed at project boards before a go-/no-go-live decision.
In the best examples, critical success factors were identified and risk
prioritised.

Lessons:
– Ensure there is formal business sign-off of deliverables against the
original project scope.
– Ensure all deliverables have acceptance criteria assigned to them.
– Ensure that non-attainment of an acceptance criterion is factored into the
go-/no-go-live decision.
– Enhance the audit trail around go-live decisions: from minuted board papers,
to acceptance criteria being met, to closure of issues and risks logs.

8. Post-go-live support

Project handover at go-live was to support functions that were not always
procedurally mature nor sufficiently resourced. In the worst cases, this led to
costs being shifted from project to production budgets and issues being resolved
with the use of temporary, expensive external resources.

Lessons:
– Ensure that production support is adequately ramped up pre-go-live
and not just activated on go-live day.

9. Documentation standards

It was important that there was evidence of key project deliverables and
milestones being signed-off as acceptable to the business. Document version
control was sometimes varied and storage control was weak. Documents required by
external stakeholders (such as external auditors and ISO9000 certifiers) were
not always readily available upon request. In one case, project documents were
lost completely in both soft copy and hard copy.

Lessons:
– Standardise project directories.
– Enhance document referencing, authorship, ownership and version control.
– Control document storage, particularly when required by third parties for
review.
– Incorporate the production of key documentation into project plans’ tasks and
go-live acceptance criteria.

Conclusions

Having a clear vision on a project – setting the scope well, sticking to it
and referencing back at closure that everything required has been delivered – is
the essence of a good project. Yet it is astonishing how often human factors and
badly executed short-cuts can warp this straightforward approach.

The temptation in implementations is for the business to see them as IT
projects, which they leave to the IT project manager, and for the IT project
manager to push through deliverables without sufficient engagement from the
business. If this happens, then, after go-live, each will blame the other for
the system’s shortcomings.

Being truly independent of both business and the project teams, internal
audit has a role in ensuring that unacceptable risks are not taken and that both
IT and the business are sufficiently and jointly engaged to produce a successful
outcome.

David Thurston has worked in IT audit at KPMG and as internal audit manager in
public and private sector roles.
Chris Kelly FCA has worked as an internal audit consultant at Ernst
& Young and as head of audit in the private sector.