The purpose of the Guidance on Assurance Reviews Process Resource Management Guide No. 106 is to provide you with an overview and practical information about Assurance Reviews:
Assurance Reviews are principle based, providing flexibility to:
- apply the information in this Guidance as appropriate to the program or project, and
- adapt to changing circumstances such as increasing financial risk and complexities associated with governance.
If you are a Senior Responsible Officer, part of a program/project team, an assurance reviewer, interviewee or a review participant, this Guidance will assist you to successfully prepare for and participate in an assurance review.
In particular, this Guidance includes a ‘Handbook for Conducting Assurance Reviews’ at Appendix C. This Handbook provides examples of areas to examine and the type of evidence that is expected at key stages throughout program and project design, implementation and delivery. The Handbook also provides reviewers with a consistent framework for undertaking Assurance Reviews across a variety of programs and projects.
Audience
The Australian Government Assurance Reviews Framework applies to some non-corporate Commonwealth entities (NCEs). This guide applies to officials of NCEs who are responsible for conducting Assurance Reviews, and Assurance Reviewers.
Key points
This guide provides an overview of the Australian Government Assurance Reviews process and assist NCEs, Assurance Reviewers and other participants to understand their roles and responsibilities.
Assurance Reviews are principle based, providing flexibility for refining and adapting to changing environments, including financial risk and complexities associated with governance.
Information in this guide is designed to be applied using common sense as relevant to the circumstances of each program/project under review.
Resources
The information in this publication is based on the Gateway Review Pack—Best Practice (Version 2), published by the State of Victoria through the Department of Treasury and Finance in 2004 and the Successful Delivery Toolkit (Version 4.5), published by the United Kingdom Office of Government Commerce (OGC), in 2004.
The Victorian Gateway documentation is subject to copyright protection by the State of Victoria and is reproduced with its permission. The Successful Delivery Toolkit is a Crown Copyright value-added product and is developed, owned and published by the OGC. It is subject to Crown Copyright protection and is reproduced under licence with permission of the Controller of Her Majesty’s Stationery Office and the OGC.
The Assurance Reviews Unit (ARU) in Finance provides a range of policy, guidance and assistance services in support of the Assurance Review function, including:
- providing guidance on the completion of the Risk Potential Assessment Tool (RPAT).
- facilitating Assurance Reviews and reviewer teams.
- assisting entities that are subject to an Assurance Review on administrative and operational matters.
- facilitating the provision of advice to Assurance reviewers related to policy enhancements and key issues.
- developing reference and supporting materials.
- periodically publishing lessons learned and better practice guidance from Assurance Reviews conducted.
This guide and related materials listed below are available on the Finance website.
Other relevant publications include:
- Resource Management Guide No. 107: Risk Potential Assessment Tool and Guidance
- Information Sheets and Questions and Answers
- Information Sheet – Assurance Reviews Process Overview
- Information Sheet – Shared Learning Building Public Sector Capability
- Q&A for Senior Responsible Officials on the Assurance Review Process
Q&A for Review Team Members on the Assurance Reviews Process.
Introduction
- This guide provides a high-level overview of each assurance process and aims to demonstrate that each process has a ‘fit for purpose’ aspect. It also outlines the circumstances and criteria that trigger each assurance process, the general timing that would apply, and where to seek further detailed information and assistance.
- Appendix C: Handbook for Conducting Assurance Reviews provides a consistent framework from which to conduct Assurance Reviews across a range of different programs and projects. The Handbook includes examples of areas to probe and the type of evidence expected at key stages throughout program/project design, implementation and delivery.
- Implementation and delivery of policy initiatives is one of the key responsibilities of Commonwealth entities. While the planning process and advice leading up to cabinet decisions are critical for effective program/project implementation, there are separate aspects of program/project delivery that need to be addressed in the implementation phase after cabinet decisions have been made.
Part 1 – Australian Government Assurance Reviews
- External assurance can add important new insights to internal control, as well as an independent perspective to support the delivery of more complex programs/projects.
- Assurance Reviews do not replace an entity’s responsibility and accountability for implementation and delivery of a program/project. Assurance Reviews are designed to strengthen assurance practices and to build capability associated with the delivery and implementation of government programs/projects and services.
- Australian Government Assurance Reviews draw on a range of proven methodologies, including the Better Practice Guide on Successful Implementation of Policy Initiatives (Australian National Audit Office and the Department of the Prime Minister and Cabinet (PM&C), October 2014) and the United Kingdom’s Office of Government Commerce, (OGC) Gateway Review Process[1].
- There are two key types of Australian Government Assurance Reviews administered by the Department of Finance:
- Implementation Readiness Assessments (IRA)
- Gateway reviews (Gateway)
- The IRA was introduced in 2011. The purpose of the IRA is to provide assurance to the responsible Minister, the accountable authority and cabinet on how well practical delivery issues are being addressed for the proposed government programs/projects.
- The government introduced Gateway in the 2006-07 Budget, focusing initially on projects that satisfied certain financial thresholds or were identified as high risk. However, complexity and implementation challenges associated with program delivery, particularly cross-portfolio programs, led the government to extend the application of the Gateway assurance methodology to apply to programs as well (2011). The purpose of Gateway is to strengthen existing governance and assurance practices, and to increase program/project management capability across government.
- Experience and feedback has shown that Assurance Reviews assist with:
- development and maintenance of robust business cases with key milestones, deliverables and benefits clearly articulated.
- implementation design and planning.
- development of risk management strategies to address challenges associated with competing priorities, resources and capability, as well as complexities associated with cross-jurisdictional responsibilities.
- management of regulatory environments that may expose a program/project to failure if not properly identified and managed.
- development of governance, accountability and reporting strategies to ensure appropriate support and oversight during implementation and delivery.
- building capability and cultivating better practice through independent peer review and monitoring.
Core principles
- Fundamental to the ongoing success of the Assurance Reviews process are its core principles, which focus on:
- providing independent assurance on how best to ensure that programs/projects are successful.
- aligning benefits to entity and government strategic objectives with clear measurable targets, timelines and owners.
- building capability through access to highly credentialed reviewers who provide mentoring and coaching.
- promulgating the lessons learned.
- Key characteristics of Assurance Reviews are:
- short duration - generally no more than five days.
- based on non-attributable interviews.
- flexibility in timing and scope, tailored to reflect the stage of policy development and delivery.
- value-add - the specialist pool of senior reviewers have skills and experience relevant to the policy delivery environment.
Variance between the IRA and Gateway review process
Review |
Timing |
Review Report |
Objective |
---|---|---|---|
IRA |
Standalone review can occur prior to a cabinet decision or soon after. |
Provided to:
|
To support the decision making process by providing government and the accountable authority with assurance on how well practical delivery issues are being addressed. |
Gateway |
Multiple reviews which occur throughout the program/project lifecycle. |
Provided to:
|
To support the successful delivery of a program/project by providing the SRO with an assessment that highlights issues that may jeopardise the delivery of benefits. |
- If a program/project is subject to both an IRA and Gateway, both processes would be integrated to minimise the potential for review burden. E.g., the IRA may replace the pre-decision review stages of the Gateway process.
Interaction with ICT Investment Approval Process and Two Stage Capital Works Approval Process
- The Assurance Reviews Framework is designed to complement the ICT Investment Approval Process and the Two Stage Capital Works Approval Process. If a program/project is subject to both Gateway and the ICT Investment Approval Process or the Two Stage Capital Works Approval Process, the requirement to conduct Gateway Gate 0 and Gate 1 reviews is not mandatory. Gateway would only commence after the approval process has been concluded or if the entity opted to participate in reviews during the approval process.
Risk management
- Better practice demonstrates that the identification and treatment of risk is undertaken at the earliest opportunity during policy design. This is an important element of the control framework necessary for effective program/project implementation.
- Risks are things that may happen at some point in the future and have the potential to negatively impact on the policy and the realisation of objectives. It is essential that risks are identified and actively managed in order to reduce their likelihood of happening or their impact on the policy or program/project.
- By identifying key factors that affect policy performance and considering how they may evolve in the future, policies can be made more robust to a range of anticipated conditions and indicators developed to identify when policy adjustments are required.
- NCEs are responsible for assessing the inherent risk factors associated with their New Policy Proposals (NPP). The Risk Potential Assessment Tool (RPAT)[2] is a standardised risk analysis tool designed to assist NCEs with this process. The resultant risk rating can inform whether additional assurance processes is recommended to government. In making this determination, Finance will consult with relevant stakeholders including the relevant entity and other central entities.
- Decisions to commission Assurance Reviews are made by government, usually during the pre-budget considerations of NPPs and Portfolio Budget Submissions. In exceptional circumstances, the Minister for Finance, in consultation with the Prime Minister and the Treasurer, can make the decision to commission an Assurance Review.
- NCEs are encouraged to consider the scheduling of their cabinet submission timetable to allow for potential pre-decision Assurance Reviews.
[1] OGG Gateway® is referred to in this document as ‘Gateway’. Gateway® is a Trademark of the UK office of Government Commerce.
[2] http://www.finance.gov.au/assurance-reviews/risk-potential-assessment-tool
Part 2 – Conducting successful Assurance Reviews
- Assurance Reviews consist of four distinct steps requiring the entity’s engagement and participation. The following sections outline the core elements of each stage and highlight some of the key elements for undertaking a successful Assurance Review.
The initiation stage
- Once an Assurance Review has been commissioned, ARU will contact the SRO and/or the Program/Project Manager to: clarify the characteristics of the program/project, discuss the timing and logistics of the review process, and discuss the skill requirements needed for the Assurance Review Team including potential reviewers.
- Following this, ARU will finalise the Assurance Review Team, including the Review Team Leader (RTL), and brief the Assurance Review Team on the program/project and their role in the review.
- At this point, the primary responsibility for coordinating the review passes to the RTL and Program/Project Manager. ARU will continue to support the conduct of the review and will be available to provide advice throughout the duration of the review.
The planning meeting
- A half-day planning meeting is held at the entity’s premises approximately two to three weeks prior to the onsite review. The planning meeting allows the SRO the opportunity to meet with the Assurance Reviews Team, discuss key issues and to brief the team about the program/project. Some of the topics that may be covered during the planning meeting include:
- policy, service delivery, legal, governance and/or contractual context of the program/project including reasons for the program/project being initiated
- relationship of the program/project with government policy, legislation and the entity’s (or entities’) outputs and outcomes
- options considered in developing the course of action
- benefits and outcomes the program/project will be expected to deliver and how they will be measured, realised and maximised
- the program/project’s status, progress to-date and planned future work
- implementation of the project plan, including an outline of the resourcing, funding and planning arrangements
- communications and change management strategies
- risks associated with the program/project, and how they will be managed.
- The Assurance Reviews Team will make requests for documentation and interviews with key stakeholders during the onsite review week.
- The Program/Project Manager is encouraged to compile a draft list of documents (using Appendices A and C as a guide) and interviewees for the Assurance Reviews Team’s consideration.
- The Program/Project Manager is responsible for organising the interviews prior to the onsite review and making the documents available to the Assurance Review Team soon after the planning meeting. The documents can be provided either electronically (i.e. Govdex) or in hardcopy. Certain classified material may only be made available on the entity’s premises.
- The planning meeting also provides an opportunity to ‘plan’ the review. This includes discussing the review agenda, resourcing requirements, and protocols for the review. E.g., prior to the onsite review, the Program/Project Manager will also need to organise facilities for the Assurance Review Team. This may include:
- a meeting room with a projector, laptop and/or access to a printer (as requested by the RTL)
- building security passes; and
- a cabinet of appropriate security rating to secure documentation.
- In addition to the above, the IRA process will also include planning meetings with central entity representatives.
The onsite review
- The onsite review is held at the entity’s premises usually over five working days. The purpose of the review is to provide the SRO with an assessment of the program/project’s progress against its stated objectives, as well as a Delivery Confidence Assessment (DCA) rating (Gateway), or an assessment of the potential issues and risks to successful implementation (IRA).
- The onsite review includes an examination of the requested documentation and interviews with key program/project stakeholders. Where possible, the Assurance Review Team is encouraged to review the documentation provided by the entity prior to the first day of the onsite review.
- Generally, on the first morning of the review week the Program/Project Manager will meet with the Assurance Review Team to clarify arrangements for the week and confirm interviews. In the case of most reviews, the first three days of the review will primarily be used for interviews and documentation review, the fourth day will be used for drafting the review report, and the fifth day will be used to finalise the review report.
- Interviews will generally take between 30-45 minutes unless otherwise requested by the Assurance Review Team. Interviews are best conducted in person, however in some cases, teleconferences may be necessary. The Assurance Review Team will usually request a short break between interviews to discuss key issues and compile notes on their findings which will form the basis of the review report.
- At the end of each day, the RTL will brief the SRO on the day’s findings. This provides the SRO with an opportunity to address any misunderstandings, progress outstanding issues or provide additional information if required. This also ensures that the review report doesn’t contain any surprises for the SRO. During an IRA, the RTL will also regularly brief ARU.
The review report
- The Assurance Review Team will commence drafting the report as soon as practicable. The report, including conclusions and recommendations, will be finalised and presented to the SRO as a draft on the penultimate day of the review by the Assurance Review Team. The reviewers will base the report on the interviews conducted and the documentation read, applying judgement and expertise. The SRO and entity will have the opportunity to provide comment on the draft report.
- On the last day of the review, the final report will be provided to the SRO by the Assurance Review Team. ARU will also be provided with a copy of the report to assist with the development of lessons learnt from Assurance Reviews. The final report, which will be signed by all members of the Assurance Review Team, will include:
- the Assurance Review Team’s assessment of overall delivery confidence (Gateway) or potential issues and risks to successful implementation (IRA)
- key findings and any recommendations, indicating when it is advisable to take action
- an overall conclusion on the program/project’s status and its readiness to progress to the next phase (Gateway)
- background to the program/project, including its origin, the outcomes it seeks to achieve, and how the outcomes link to the entity’s business strategy and/or high level policy objectives
- the purpose, scope and approach of the review, logistics of the review, including review dates, SRO, Assurance Review Team membership, stakeholders interviewed and documents reviewed; and
- the progress achieved against previous review recommendations (if applicable).
- Gateway review reports are for the purpose of informing the entity and as such, they are only provided to the SRO. SROs are encouraged to circulate their reports with key stakeholders and governance arrangements. The aim is to ensure that the appropriate people are aware of issues arising and problems identified, and are able to take the requisite action[3].
- Sharing and reviewing outputs of assurance activities underpins an effective integrated assurance model, and maintaining an integrated assurance log can be a useful tool. Entities are encouraged to escalate the outcomes and recommendations from assurance to the level where appropriate remedial actions can be sanctioned.
- IRA review reports are provided to the responsible minister, the portfolio secretary and/or accountable authority, the SRO, Finance, PM&C and the Treasury. Finance will also refer to the outcome of the IRA in a briefing provided to government.
- Gateway reports provided to accountable authorities are subject to the Freedom of Information Act 1982 (FOI Act). The entity has responsibility for dealing with FOI requests for Gateway reports. Other information held by entities or by ARU related to a Gateway review may also be subject to the FOI Act.
Key elements for a successful Assurance Review
Planning and coordination
- Successful Assurance Reviews are underpinned by thorough planning and careful coordination. Throughout the review process, discussions are held with key participants from the entity, ARU and the Assurance Review Team to clarify the intent, requirements and timeframes of a review to assist with the planning and coordination process.
Open and clear dialogue
- During each review, access to key participants and stakeholders is integral to the success of the review. Successful reviews will have fully engaged participants and stakeholders who demonstrate a willingness to share information openly and honestly. Assurance Reviews are most effective if those involved are conscious that the review is being conducted to provide assistance to the program/project, rather than being seen as an externally imposed assessment or compliance audit.
- Participants are encouraged to respond to questions from the review team accurately and completely. This includes communicating their role, responsibilities and reasons for their actions and decisions as requested. Participants are encouraged to make themselves available to answer follow-up questions and provide additional documentation in a timely manner.
- Reviewers are encouraged to put participants and stakeholders at ease, give them confidence to talk openly about the program/project and contribute their thoughts, concerns and issues. Reviewers are encouraged to employ a positive, constructive, strictly objective, consistent and non-personal manner when questioning participants. Review teams are also encouraged to clarify the meaning of the terminology used with the program/project team to ensure an accurate and useful review outcome.
- Review reports will be written on a non-attributable basis.
Confidentiality and accountability
- Fostering confidentiality during the review process encourages candour and helps ensure access to information. As part of the confidentiality, it is critical that all program/project documentation is returned to the entity at the conclusion of each review. The Assurance Review Team’s notes and draft reports will need to be destroyed on the last day of the review. The only permanent record of a review will be the final review report.
- To ensure an appropriate level of awareness of an Assurance Review, SROs are encouraged to keep their accountable authority informed of the progress of the review, including outcomes and recommendations. It is considered better practice for the accountable authority and where appropriate, the responsible minister to be informed of significant review conclusions.
- Participation in Assurance Reviews does not absolve an entity’s accountability for their programs/projects. SROs are responsible for determining and implementing any actions required to address recommendations critical to the success of a program/project.
Work health and safety requirements
- In addition to complying with specific health and safety contract requirements, reviewers will need to take reasonable care of their own health and safety and ensure that their actions or omissions do not adversely affect the health and safety of others.
- Reviewers will need to comply with reasonable instructions given by the entity on whose premises they are working. Reviewers have the right to cease unsafe work on their own initiative when there is a serious risk to safety or when there is reasonable concern for safety. Such circumstances will need to be discussed with the entity and ARU without delay.
- The entity is required to provide a ‘safety induction’ for the reviewers so that they know how/where to report incidents and understand the emergency evacuation procedures etc.
Part 3 – Roles and responsibilities
Entities
Senior Responsible Official
- The SRO is responsible for the successful delivery of the program/project. This will often be an official at the Senior Executive Service band two or three level, who has the authority to make decisions affecting the program/project but who is not responsible for its day-to-day management.
- During an Assurance Review, the SRO’s responsibilities include:
- providing input to ARU on the skill requirements of the Assurance Review Team
- briefing the Assurance Review Team on key aspects of the program/project during the planning meeting
- assisting the Assurance Review Team to gain access to key stakeholders and documentation
- receiving daily briefings from the Assurance Review Team during the onsite review
- ensuring appropriate action is taken to address the review findings; and
- providing feedback to ARU on the quality of the completed Assurance Review and the Assurance Review Team.
Program/Project Manager
- The Program/Project Manager has primary responsibility for the day-to-day management of the entire program/project. During an Assurance Review, the Program/Project Manager will assist the Assurance Review Team to obtain a thorough understanding of the program/project, including:
- the program/project origin, stated outcome(s) and how benefits will be realised
- links between stated outcomes and the entity’s business strategy and/or high-level policy outcomes
- the program/project objectives, desired outputs and delivery strategy; and
- the governance, composition and management of the program/project.
- The Program/Project Manager, or their nominee, must ensure that:
- documents requested by the Assurance Review Team are made available shortly after the planning meeting via Govdex or as agreed with the Assurance Review Team
- any additional documentation or highly classified material is provided to the Assurance Review Team either prior to, or during the onsite review, at the entity’s premises
- interviews are organised with key stakeholders during the onsite review, and interviewees are informed about the purpose and logistics of the interview; and
- suitable meeting rooms and facilities are available to the Assurance Review Team for interviews and meetings during the onsite review.
- In addition to program/project roles, when assuring agile delivery of a digital service the following roles are recommended to also be involved:
- Service manager - accountable for all aspects of the current and future service, including the non-digital channels. They are the keeper of the Service Vision that will guide the team to deliver a service aligned to that vision. The service manager is responsible for ensuring that the service meets user needs, and is responsible for managing channel shift to the digital channel.
- Product manager - a service may be broken down into a number of products. Each product manager is responsible for the effective development of their specific product, i.e. meeting user need both in itself and as part of the overall service.
- Delivery manager - responsible for ensuring that teams clearly understand what is expected of them and that sufficient resources are available to effectively deliver all products/services.
Other key stakeholders
- Other stakeholders include any individual or entity that is either potentially affected by, or who has potential effect on the program/project. This may include central entities, other entities, interest groups or private sector bodies. Other stakeholders may be asked to meet with the Assurance Review Team and provide relevant information in its entirety and in a timely manner.
Assurance Reviews Unit
- The Department of Finance’s ARU provides guidance, support and additional information on Assurance Reviews to Assurance Review Teams and entities as required.
- ARU does not undertake Assurance Reviews. Assurance Reviews are coordinated by ARU but are undertaken by an independent review team appointed by Finance with the appropriate security clearance and experience.
- ARU’s responsibilities include:
- providing a point of contact to schedule and coordinate Assurance Reviews
- maintaining a pool of Assurance Reviewers
- assessing and appointing suitable Assurance Reviewers
- communicating the requirements of Assurance Reviews to entities and assisting them as required; and
- compiling and disseminating non-attributable lessons learned of major program/project management to assist entities to implement practices and controls that increase the likelihood of successful program/project outcomes.
- ARU facilitates Assurance Reviews by:
- liaising with the SRO regarding the skills required for the Assurance Review Team
- assembling the Assurance Review Team and assisting with logistical and administrative arrangements for the planning meeting
- briefing the program/project team on the requirements of an Assurance Review
- providing the Assurance Review Team with relevant templates
- responding to queries and providing advice to the entity and the Assurance Review Team as required
- ensuring that procedural requirements have been met
- collating evaluations on the Assurance Review Team’s performance; and
- analysing review reports and recommendations to identify non-attributable lessons learned.
Reviewers
- Assurance Reviews are conducted by an independent Assurance Review Team appointed by Finance. An Assurance Review Team usually consists of a RTL and up to three Review Team Members (RTMs).
- Reviewers may be sourced from the public or private sector. Public sector reviewers have the unique and strategic learning opportunity to work across government and contribute their experience to provide assurance to important programs/projects. It is important to note that public sector reviewers are selected for their expertise, and not to represent their entity.
- Similarly, private sector reviewers are selected for their expertise, not to represent their firm, and may not use the Assurance Reviews process to actively solicit business for themselves or their firm.
Reviewer selection
- When selecting reviewers, ARU will consult with the entity to ensure an optimal mix of reviewers is selected for the team. Consideration will be given to a number of factors, including:
- potential conflicts of interest
- reviewer knowledge, skills, experience relevant to the program/project and the review to be undertaken (refer to Appendix B)
- reviewer availability; and
- the level of security clearance required (if applicable).
- Continuity of team members from one Gate/Stage to another ensures consistency and understanding of the program/project under review. Each Assurance Review Team usually includes one or two members from earlier reviews, however there may be instances where this is not possible.
Review Team Leader
- The RTL’s role is essential to the success of an Assurance Review. They are primarily responsible for facilitating communication and relationships with the SRO, the program/project team, the Assurance Review Team and other review participants. The RTL’s responsibilities include:
- liaising with the SRO prior to the planning meeting to introduce themselves, establish the context and logistics of the review and identify concerns and expectations
- contacting each member of the Assurance Review Team prior to the planning meeting to introduce themselves and develop an understanding of the skillsets and strengths across the team
- identifying key stakeholders to be interviewed, relevant documentation needed for the review, and potential information access constraints
- formulating the protocols and logistics for the review, and ensuring that all participants understand their responsibilities and the purpose and structure of interviews
- acting as the chair during the planning meeting and the interviews, and ensuring that the Assurance Review Team focus on issues that are important to the program/project’s success rather than the technical aspects
- providing daily briefings to the SRO during the review on the day’s findings
- providing leadership to the RTMs, including coaching, support and feedback to develop their skills; and
- providing an evaluation on each RTM to ARU to assist in assembling future Assurance Review Teams.
Review Team Members
- RTMs are selected for the relevance of their skills and experience. RTMs’ responsibilities include:
- identifying key stakeholders to be interviewed and relevant documentation needed for the review
- considering documentation relevant to the review and forming an opinion on the adequacy of the documentation, based on their experience and expertise
- participating in the discussion and assessment of the program/project under review in a professional and objective manner
- assisting the RTL to conduct interviews and gather/analyse the information made available
- assisting to write and contributing to the development of a high-quality and constructive review report
- working cooperatively with all the review participants, including the Assurance Review Team, the entity and ARU
- contributing, as required, to briefing the SRO on the review’s findings, conclusions and recommendations; and
- providing an evaluation on the RTL to ARU.
Part 4 – Implementation Readiness Assessment (IRA) methodology
- IRAs are short, independent reviews that provide assurance to government for high risk policy proposals. An IRA is usually commissioned by the Expenditure Review Committee (ERC) on the basis of the risks of the proposal and advice from Finance.
- IRAs are generally focused on reviewing proposals prior to seeking a government decision, however, in some circumstances it is not logistically possible to conduct an IRA in time to inform government deliberations. In these cases, an IRA may still be applied, with the review findings to be included in a letter from the Minister of Finance to the Prime Minister, and copied to the relevant portfolio Minister and the Treasurer.
- The IRA is a collective and collaborative effort to examine an issue from different points of view prior to a decision. IRAs are designed to:
- assess the program/project implementation strategy against its specific objectives
- provide early identification of any areas that may require corrective action
- provide a point in time assessment of the effectiveness in planning for implementation
- assist to strengthen policy design by building recognition of common values, shared commitment and emerging issues, and by providing an understanding of causal relationships
- increase confidence that a program/project implementation strategy is effective and conclusive, draws on experience and better practices, has effectively identified risks to implementation and has appropriate mitigation strategies in place; and
- ensure that central entity stakeholders have the opportunity to inform the IRA review.
- The IRA is a structured, time-limited review, relying on interviews and existing documentation/material, and is not anticipated to be unduly onerous for entities in terms of administration or preparation.
- The outcome of an IRA is a succinct review report which clarifies key issues and risks, and provides insights for managing and mitigating implementation risks.
- The IRA process draws on the methodology of:
- Better Practice Guide on Successful Implementation of Policy Initiatives[4]
- “Chapter 9: Lessons Learned, Identifying the challenges to implementation during policy development”, of Performance Audit No.12 2010-11, Home Insulation Program (Australian National Audit Office, October 2010)[5]
- The United Kingdom’s Office of Government Commerce (OGC), Gateway Review Process.
Implementation Readiness Assessment ratings
- The following three-tiered IRA rating scale differs from that applied to the Gateway methodology. Given the nature of the IRA review, the indicator applied for the IRA reflects the potential issues and risks to successful implementation of a proposal.
Assessment |
Description |
---|---|
Green |
Successful implementation of the program/project appears likely and there are no major outstanding issues/risks that appear to threaten implementation significantly. |
Amber |
Successful implementation of the program/project appears feasible but prompt management attention is required to address issues/risks. |
Red |
Successful implementation of the program/project is in doubt with significant issues/risks already apparent. Urgent action is required to address these. |
Key focus areas
- The IRA process focuses on the following key focus areas to determine how well an entity is planning for future implementation of a policy initiative:
1. Policy design |
The key question is whether the means of implementing/delivering this policy is likely to work in practice—whether it ‘hits the mark’. During policy design, in order to move forward without absolute information, assumptions need to be made. In a NPP, there may be a high degree of unknown elements of the proposal. If the assumptions made about an initiative are clearly identified, along with their sensitivity to change, then Ministers and those implementing the initiative can be better informed of the possible risks and their consequences. |
2. Implementation planning |
A policy initiative is more likely to achieve the best possible outcomes when the question of how the policy is to be implemented has been an integral part of policy design. Has sufficient consideration been given to major implementation risks, e.g., has enough time been spent on planning for implementation, what parties will contribute to implementation and have they been involved during the policy development stage? |
3. Governance arrangements |
While accountable authorities are commonly required to deal with an array of policy, program/project and organisational issues, it is also important that ongoing attention is given to measures to reinforce good governance and effective administration. Implementation of policy is more likely to succeed if there is strong executive-level support for the delivery processes for the policy. |
4. Risk management |
The increased understanding across the public sector of risk management processes does not in itself guarantee the proper treatment of challenges to successful policy implementation. The effective management of risk requires a robust, entity-wide risk management climate where decisions are based on accurate and well informed judgements. An important consideration for the accountable authority is the availability of appropriately skilled senior personnel to manage key risks to successful implementation |
5. Stakeholder management and communications |
Stakeholder management starts with a clear objective for consultation, followed by an identification of the range of people and entities with an interest in the initiative. Have the right stakeholders been identified, has sufficient consideration been given to how stakeholder interactions will be managed during the implementation phase, and is there clear accountability for stakeholder management, including managing expectations? |
6. Evaluation and performance |
Evaluation and performance is vital to measuring the success of policy implementation. Will the policy initiative be formally evaluated, do data collection arrangements support effective evaluation, how will results inform future implementation, have key performance indicators been developed and how will they be applied in ongoing program management? |
- Each focus area is rated using the following rating scale. The summation of all the issues raised in each focus area and the risks they pose to successful delivery are then summarised in the body of the report. This establishes an overall assessment on whether the program/project can be implemented successfully.
Assessment |
Description |
---|---|
Low |
There are no major outstanding issues/risks in this key focus area that at this stage appear to threaten implementation significantly. |
Medium |
There are issues/risks to implementation in this key focus area that require prompt management attention. |
High |
There are significant issues/risks in this key focus area that may jeopardise successful implementation. |
Report recommendation categories
- The recommendations made within IRA reports are critical for both highlighting strategies required to address potential issues/risks, and for establishing the priority of the key actions to be taken. The Assurance Review Team provides recommendations which are prioritised in terms of urgency using the following categories:
Assessment |
Description |
---|---|
Critical (Do now) |
To increase the likelihood of a successful outcome it is of the greatest importance that the program should take action immediately. |
Essential (Do by) |
To increase the likelihood of a successful outcome the program should take action in the near future. Whenever possible essential recommendations should be linked to program milestones (e.g. before contract signature and/or a specified timeframe i.e. within the next three months). |
Recommended |
The program should benefit from the uptake of this recommendation. If possible recommendations should be linked to program milestones (e.g. before contract signature and/or a specified timeframe i.e. within the next three months). |
Part 5 – Gateway review process (Gateway) methodology
- Gateway reviews examine programs/projects at key decision points during design, implementation, and delivery. Gateway aims to provide independent, timely advice and assurance to the SRO as the person responsible for delivering the program/project outcomes.
- Gateway is not an audit process nor does it replace an entity’s responsibility and accountability for implementing decisions and programs/projects. Gateway plays a unique role in strengthening assurance practices, as well as building and sharing capability associated with the delivery and implementation of government programs/projects.
- The Gateway process is intended to be supportive and forward-looking, taking into account future plans to deliver, and intended outcomes and benefits. The use of highly skilled and experiences reviewers, sourced from private and public sectors is intended to increase the confidence in implementation. Gateway reviews assist the SRO’s oversight and governance of major programs/projects, and assist with the delivery of agreed programs/projects in accordance with the stated objectives.
- Key attributes include:
- time limited reviews (generally five days) at key decision points or milestones (referred to as ‘gates’ for projects, and ‘stages’ for programs’)
- reviews that can be conducted multiple times and on a regular basis through-out the life of the program/project
- principles based to be flexible and adaptable to changing environments
- a point-in-time assessment of delivery confidence; and
- cultivating a benefits led approach throughout program/project design, implementation and delivery.
- For the best results, a review is carried out shortly before a decision point or stage transition to allow sufficient time for any recommendations to be implemented.
- Gateway applies to proposals which require government approval, are assessed to be high risk, and satisfy the following financial thresholds:
- projects with a total cost estimated to be $30 million or more for procurement or infrastructure; or
- projects with a total cost estimated to be $30 million or more, including an ICT component of at least $10 million; or
- programs with a total cost estimated to be greater than $50 million.
Note: Gateway does not apply to Defence Capability Plan projects assessed by cabinet under the Kinnaird two-pass approval process.
Gateway for projects
- There are six different reviews that occur at critical stages (or Gates or decision points) of a project’s lifecycle. They are:
Critical Stage |
Focus |
---|---|
Gate 0 |
Business need - assures that the scope and purpose has been adequately assessed, communicated, fits within the entity's overall business strategy and/or whole-of-government strategies and policies and that the expected benefits have been identified and measures have been considered. |
Gate 1 |
Business case - focuses on the robustness of a project's proposed approach to meeting the business requirement and can be delivered within the timeframe and with the resources provided. Assures that a benefits management approach has been applied, improvements are clearly defined and can be quantified. |
Gate 2 |
Delivery strategy – provides assurance that the procurement strategy establishes a clear definition of the project and a plan for its implementation, has made an assessment of the project’s potential for success and benefits agreed upon in previous stages have been aligned to the delivery effort, and if the project is ready to invite proposals or tenders. |
Gate 3 |
Investment decision - provides assurance on the supplier selection that the business needs are likely to be met through the project and contract management controls, and that the processes are in place for contract delivery. Assures that benefits management strategies and plans have been incorporated. |
Gate 4 |
Readiness for service - provides assurance on whether the solution is robust before delivery, assesses organisational readiness before and after delivery, and considers the basis for evaluating ongoing performance and whether benefits are likely to be achieved. |
Gate 5 |
Benefits realisation - focuses on measuring the project’s success to date in achieving its objectives, expectations for the future and building in remedial action to deal with any potential risks. |
Gateway for programs
- There are three different review stages that occur for programs. They are:
Critical stage |
Focus |
---|---|
First stage review |
Conducted before or soon after government approval. It assists entities in defining the program by examining the business need and formulation of the business case. Can also be conducted whenever the priority or the scope of the program changes significantly. |
Mid stage review |
Focus is on assessing the program execution with the number of these reviews being determined by the complexity, timeframe and risks attached to the program. |
End stage review |
Focus is on program closure, including program controls, records management and the identification and application of lessons learned as well as the delivery of the intended outcomes and benefits. |
Blended Gateway reviews
- Programs do not always fit into the Gateway structure for projects (Gate 0 to 5) because they can represent a series of interrelated projects with a common objective, or a broad framework or policy concept that may result in a series of largely independent smaller projects (potentially all at different stages of implementation).
- The ‘blended’ review approach helps to reduce the review burden on entities while simultaneously providing program strategic alignment and project milestone delivery assurance.
- Identifying the critical factors in a multi-project program, including the issues that need to be addressed to realise benefits, is an important component of the blended review approach.
- Program reviews will assess the significance of any project to the overall success of the program, and where the single project could benefit from a focussed review, the overall program review will accommodate a combined focus of program and project review. E.g., blend Gate 1 with a First Stage Review or blend Gate 4 with a Mid Stage Review.
Combined Gateway reviews
- Combined Gateway reviews (i.e. combined Gate 0 and 1) may be conducted when a project has reached the point in its lifecycle where the issues relevant to the subsequent Gateway milestone are pertinent at that point in time.
Phased Gateway reviews
- Phased Gateway reviews can be conducted, e.g. a Gate 2a followed by a Gate 2b, where the entity, ARU and/or the RTL considers that this approach would add value and be appropriate for the project.
- A phased approach would work well, e.g., where there are discrete phase ‘releases’ that would benefit from targeted assurance. This is often the case for large ICT projects and projects applying an Agile methodology. A phased approach may also be applied where the SRO requires independent assurance before committing to a course of action prior to the timing of the next scheduled review.
Intermediate assessments
- A significant lag can sometimes occur between Gateway reviews. This is most obvious in complex ICT and construction projects – during the critical ‘build’ stages of implementation (between Gate 3 and Gate 4).
- Intermediate assessments are based on the principles existing in the Gate 0 – Business need review, and provide entities with interim assurance focussed on strategic alignment, the strength of the business case, and effort to optimise benefits to government. Additionally, intermediate assessments can assist to highlight key risks and reconfirm stakeholder commitment.
- The timing of intermediate assessment reviews will take into consideration planned reviews and key decision points, ensuring that no more than 18 months’ elapse between reviews. This promotes better implementation by entities and provides the opportunity for earlier intervention by government where delivery significantly slips.
Delivery Confidence Assessments (DCAs) for Gateway
- The DCA represents the collective view of the Assurance Review Team on the overall likelihood of success for the program/project, taking into account the timing of the review and the stage within the program/project lifecycle. Delivery Confidence is the confidence in the project or program’s ability to deliver its aims and objectives:
- within the timescales
- within the cost envelope
- and to the quality requirements including the delivery of benefits, both financial and non-financial
- all as laid down in the most recent formally approved mandating document (e.g. NPP, implementation plan or business case).
- The following five-tiered rating system provides a level of granularity to help entities focus on emergent issues and appropriate mitigation strategies:
Assessment |
Description |
---|---|
Green |
Successful delivery of the program/project to time, cost, quality standards and benefits realisation appears highly likely and there are no major outstanding issues that at this stage appear to threaten delivery significantly. |
Green/Amber |
Successful delivery of the program/project to time, cost, quality standards and benefits realisation appears probable however constant attention will be needed to ensure risks do not become major issues threatening delivery. |
Amber |
Successful delivery of the program/project to time, cost, quality standards and benefits realisation appears feasible but significant issues already exist requiring management attention. These need to be addressed promptly. |
Amber/Red |
Successful delivery of the program/project to time, cost, quality standards and benefits realisation is in doubt with major issues apparent in a number of key areas. Urgent action is needed to address these. |
Red |
Successful delivery of the program/project appears to be unachievable. There are major issues on program/project definition, schedule, budget, quality or benefits delivery. The program/project may need to be re-baselined and/or overall viability re-assessed. |
Key focus area assessment ratings
- The Assurance Review Team will also provide an assessment against each of the “key focus areas” probed using the following rating scale.
Assessment |
Description |
---|---|
Green |
There are no major outstanding issues in this key focus area that at this stage appear to threaten delivery significantly. |
Amber |
There are issues in this key focus area that require timely management attention. |
Red |
There are significant issues in this key focus area that may jeopardise the successful delivery of the project. |
Report recommendation categories
- The recommendations made within Assurance Review reports are critical for both highlighting strategies required to address potential issues, and for establishing the priority of the key actions to be taken. The Assurance Review Team provides recommendations which are prioritised in terms of urgency using the following categories:
Assessment |
Description |
---|---|
Critical (Do now) |
To increase the likelihood of a successful outcome, it is of greatest importance that the project take action immediately |
Essential (Do by) |
To increase the likelihood of a successful outcome, it is important that the project take action in the near future. Whenever possible, link essential recommendations to project milestones (e.g. before contract signature and/or a specified timeframe i.e. within the next three months). |
Recommended |
The project would benefit from the uptake of this recommendation. If possible, link recommendations to project milestones and/or a specified timeframe. |
Previous recommendations and action taken
- The SRO is responsible for implementing recommendations and taking remedial action. The SRO is encouraged to maintain an official record of how review recommendations have been implemented, or why recommendations haven’t been implemented.
- Prior to a review, the entity will complete the ‘Actions Taken’ section of the review report, demonstrating the remedial actions taken in response to the previous review recommendations.
- In reviewing the actions taken, the Assurance Review Team will indicate in the ‘Review Team Comments’ section whether recommendations have been addressed (as defined below), and where appropriate, provide further comments to explain the action taken.
- Fully – the recommendation has been fully implemented by the entity.
- Partially – the recommendation has been partially implemented by the entity.
- Not Addressed – the recommendation has not been implemented by the entity.
Enhanced Notification process
- An Enhanced Notification (EN) process is in place so that early remedial intervention can occur if a program/project is experiencing significant problems.
- The EN is a staged escalation process which involves the Finance Secretary writing to the relevant accountable authority to advise that the Assurance Review Team has raised concerns, which may affect the likelihood of program/project achieving the intended outcomes and benefits. This advice, which includes notification of all recommendations made in the Gateway review report, asks the entity to consider appropriate escalation action.
- The EN process applies throughout the program/project lifecycle, and is triggered by incidences of Red or sequential Amber or Amber/Red DCA ratings. The EN process involves the following three levels of escalation:
- First level – triggered by one Red or two sequential Amber or Amber/Red DCA ratings.
- The Secretary of Finance will write to the relevant NCE accountable authority escalating awareness that the program/project has issues to address and that early rectification is required.
- While it is better practice to advise the responsible Minister/s, it would be a matter for the NCE accountable authority to decide.
- Second level – triggered by two sequential Red DCA ratings, or three sequential Amber or Amber/Red DCA ratings.
- The Secretary of Finance will again write to the NCE accountable authority advising that the issues remain.
- The NCE accountable authority is required to seek an action plan from the SRO within 30 business days, advising how the issues will be addressed.
- The NCE accountable authority is required to inform the responsible Minister/s and the Secretaries of PM&C and Finance.
- If the Secretary of Finance considers that the issues are significant enough to warrant stronger intervention, the letter may also suggest that the NCE accountable authority commence an independent review to assist them to decide on how to proceed.
- Third level – triggered by three sequential Red DCA ratings, or four sequential Amber or Amber/Red DCA ratings.
- The Secretary of Finance will write to the NCE accountable authority for the third time, advising that very significant problems continue to exist.
- The NCE accountable authority is required to undertake an independent external review immediately.
- The NCE accountable authority is required to inform the responsible Minister/s and the Secretaries of PM&C and Finance on the action being taken following receipt of the third letter.
- First level – triggered by one Red or two sequential Amber or Amber/Red DCA ratings.
- The independent review in the second and third level of the EN process can refer to either an internal or external review, conducted by reviewer/s independent of the program/project and outside of the governance framework. The purpose of the review is to provide an expert assessment of the program/projects progress and to recommend course corrections.
Action plan
- The second level of the EN process requires the accountable authority to seek an action plan from the SRO within 30 business days (of receipt of the Secretary of Finance’s letter) to address the issues raised in the review.
- Action planning is the process that guides the activities of a program/project. It is the process of planning what needs to be done, when it needs to be done, by whom it needs to be done, and what resources or inputs are needed to do it.
- In the context of the EN process, the action plan ensures that key issues which affect the DCA are being effectively addressed. An action plan may include (but is not limited to) the following elements:
Rationale |
A statement of what must be achieved, the purpose, objective/s and the context which brought about the need for the action plan. This would also include the recommendations made in the preceding Gateway review which initiated the need for the action plan. |
Action proposed |
Actions proposed to mitigate the issues raised by the preceding Gateway review. Being specific about what the entity is trying to achieve makes it easier to undertake corrective or maintenance actions and evaluate success. |
Accountability |
It is important to nominate an Action Officer responsible for each action, championing the specific actions and ensuring outcomes are met. Once completed, the action plan would need to be endorsed by the accountable authority. |
Budgeting/Resourcing |
Where applicable, consider any resources required for the actions proposed. |
Timing |
Ensure that you consider when the activity needs to be in place and for how long it is required. Also provide a current status report of the proposed actions. |
Challenges/Risks associated in achieving intended outcomes |
Identify the challenges/risks associated with the proposed actions, what is their likely impact and how will they be managed/mitigated. |
Stakeholders/ Consultation |
Identify key stakeholders and how/when key stakeholders will be consulted. |
Action/Progress update |
The action plan is a living document and would be regularly reviewed ensuring that actions are on track and emerging issues are managed. |
Evaluation/Performance indicator |
Evaluation is an opportunity to examine how well you are implementing the activities, to take stock of the progress, and to formulate lessons learned. How will you know what success looks like? |
Assurance of Action Plan (AAP)
- Entities may be offered an Assurance of Action Plan review at the second stage of the EN process.
- An AAP review is an optional (at the SROs discretion), one-day onsite review, led by the RTL from the previous Gateway review, aimed at providing constructive and timely assistance to the SRO to finalise the entity’s action plan. The Assurance Review Team will provide a confidential report to the SRO of the entity and ARU.
- The action plan together with the previous Gateway review recommendations form the terms of reference for the AAP. The object of the AAP is to:
- assist the SRO to better understand the issues raised during the previous Gateway review
- provide input to the remedial action plans put in place to get the program/project implementation back on track; and
- provide an early opportunity for entities to receive assurance that their action plans address the issues identified in the previous Gateway review.
- AAP reviews are generally concluded within two days, inclusive of a planning meeting and the final report handover.
1. Initiation stage
- Once an EN letter has been signed, a meeting can be organised between ARU and the SRO (at the discretion of the SRO). The purpose of the meeting is to clarify the purpose of the AAP review, discuss the AAP Assurance Review Team membership, and discuss the timing and logistics of the review.
- Following the meeting, ARU will finalise the Assurance Review Team, confirm timing and logistics of the review, and brief the Assurance Review Team on their role in the AAP review.
- ARU will continue to support the conduct of the review and be available to provide advice throughout the duration of the review.
2. Planning meeting
- Prior to the AAP review, the RTL will contact the RTM/s, the Program/Project Manager and the SRO. This could be a formal planning meeting or a teleconference to discuss:
- the program/project’s progress since the last Gateway review
- the progress of the action plan; and
- the list of interviewees and the documents that will be required during the onsite review.
- Where a brief planning meeting takes place, it is good practice for the meeting to be conducted within a reasonable time, i.e. within two weeks prior to the onsite review.
3. Onsite review
- Evidence gathering for an AAP review is normally conducted during a one-day onsite review. The confidential AAP report is prepared on the review day, in consultation with the SRO and the entity’s program/project team and stakeholders.
- At the conclusion of the review day, the Assurance Review Team meets with the SRO to present the draft report and discuss its contents. Any changes/updates to the report can be finalised the following day before the final report is submitted to the SRO. A copy of the final report is also provided to ARU to identify lessons learned and evidence of best practice and provided to subsequent Assurance Review Teams as pre-reading material.
Appendix A: Example document list required for a Review
- Program/project overview including objectives, key policy assumptions, background material, press releases
- Key review(s) that underpin the policy proposal
- Program/project budget documentation i.e. cabinet submission/NPP, costing templates including staffing resources
- Statistical and/or research material used in the development of the program/project policy
- Implementation plans
- Benefits management strategy
- Program/project timeline, showing critical path, dependencies and key milestones
- Risk matrix and risk management approach
- Performance measures and reporting regime
- Evaluation and communication plan
- List of other entities involved in the program/project
- List of stakeholders
- List of relevant files
- Details of the consultative process showing who was consulted and key submissions/discussion papers
- Governance model including minutes from any steering or program/project management committees
- Issues log
- Organisation chart for relevant areas of the entity
- Details of the implementation team
- List of entity contacts and phone numbers for the program/project
Refer to Appendix C for specific document lists for each Assurance Review Process.
Appendix B: Skills profile of an Assurance Reviewer
ARU maintains a register of potential Assurance Reviewers, including information on their experience and qualifications relevant to Assurance Reviews. RTLs and RTMs will be matched to reviews based on their skills and experience, and the requirements of the review.
Potential reviewers may wish to provide information to ARU on their skills, expertise and experience in the following areas to be considered for future reviews include:
- Public sector senior executive experience
- Portfolio/program/project management expertise and experience such as:
- program/project/portfolio management
- business analysis
- business change management
- risk management
- benefits realisation
- Business and policy expertise and experience such as:
- policy design and implementation
- financial analysis
- procurement
- legal and contracting
- ICT expertise and experience such as:
- IT project management
- systems integration
- enterprise architecture
- system delivery and management
- electronic documents and records management systems
- Infrastructure expertise and experience such as:
- construction / property project management
- facilities management
- Operational expertise and experience such as:
- operations support experience
- service delivery experience
- government program /project and grants administration
- Specialised skills or qualifications or experience such as:
- Public Private Partnerships
- logistics
- Gateway reviews undertaken in other jurisdictions
- other relevant skills or qualifications.
Appendix C: Handbook for conducting Assurance Reviews
This Appendix is designed to complement the expertise of Assurance Reviewers and the information outlined in the body of this guide, providing a consistent framework from which to conduct Assurance Reviews across a range of different programs and projects. It contains examples of areas to probe and the types of evidence expected at key stages throughout program/project design, implementation and delivery.
A well-conducted review focuses on areas that are critical to a program/project’s success while ensuring that the review is thorough. In order to achieve this, Assurance Review Teams are encouraged to:
- conduct reviews in an organised way, covering, but not limited to, the issues identified in this Appendix and adhering to the key steps in the review process as outlined in the body of this guide
- be united in purpose, following the principles and objectives of the Assurance Review process, not unnecessarily focused on any individual RTM’s concerns
- involve participants in professional, focused, and harmonious peer discussions, and avoid a confrontational, inquisitorial approach to find problems and mistakes
- recognise and respect examples of good practice in the work done to date; and
- provide recommendations that are clear and action-oriented.
Gateway for projects
Gate 0 – Business need review
Purpose
A Gate 0—Business need review is a broad, strategic review that occurs at the start-up stage of a project to inform decision-making.
In a broader sense, this type of review provides assurance to the SRO that the scope and purpose of the project has been adequately accessed and communicated to stakeholders, and fits within the entity’s overall business strategy and/or whole-of-government strategies/policies. It also aims to test whether stakeholders’ expectations of the project are realistic in relation to planned outcomes, resource requirements, timetable and achievability.
A Gate 0 review occurs when the preliminary justification for the project is drawn together. It is based on a strategic assessment of business needs, an analysis of the stakeholders whose commitment is needed to achieve the objectives, and a high level assessment of the project’s likely costs and potential for success. In this case, a Gate 0 review comes after the business need has been identified but before a proposal goes forward for approval.
Better practice identifies that a Benefits Management approach be applied at the start of the policy and maintained all the way through the life of the new capability. Benefits management is a key part of strategic delivery of business change. Applying a benefits approach assists the justification of the initiative by clearly showing where the expected improvements are to be found.
At this stage the NPP and business case are developed. Entities are encouraged to include benefits statements in the implementation plan supporting the NPP. The benefits statement provides a clear description of the intended beneficiaries and expected benefits of the policy measure[6].
Review Team expectations in Gate 0 reviews
- Ensuring that the project contributes to the entity’s business strategy and to high-level government policy objectives and outcomes
- reviewing the business need to determine whether it be structured as a single project or a program of smaller component projects
- confirming that the project’s potential to succeed has been considered in the wider context of the entity’s delivery plans
- checking that there has been an assessment of the market’s capacity to achieve the required outcome
- ensuring that benefits analysis has taken place
- ensuring the project is clearly understood and supported by users and stakeholders
- reviewing the arrangements for leading, managing and monitoring the project
- reviewing the arrangements for identifying and managing the risks of the project, including external risks such as changing business priorities
- where the Gate is integrated with the ICT Investment Approval/Two Stage Capital Works Approval processes, ensuring that the review supports development of the first pass business case
- checking that financing has been adequately assessed for the project; and
- checking that the project is realistic, properly resourced and has authorised work plans through to the next stage of the project.
Documents required in Gate 0 reviews
In order to undertake a Gate 0 review, the Assurance Review Team may require access to the following documentation:
- the NPP or cabinet submission/decision (if applicable).
- the NCE’s Portfolio Budget Statement, business plan or equivalent documents. These documents set out the entity’s outcomes and outputs, strategic direction, high level aims and objectives.
- a project brief that provides information about:
- objectives—a high-level description of the purposes, outcomes sought, key deliverables and timelines,
- critical success factors—the main criteria against which the success of a project will be measured. This could include a benefits management plan showing that the project is required, the expected benefits and the business changes and enablers that will be required for the anticipated benefits (these could be linked to the objectives),
- background—an outline of the key drivers for the project, showing how it will contribute to policy outcomes and/or the business strategy
- scope—a high-level view of the boundaries of the project, the main assumptions, possible constraints and dependencies
- stakeholders —and how they will be engaged
- finance—the assessed costing and budget for the project and its components
- organisation—the way in which the project is to be organised, led, managed and monitored
- risks—the main risks identified so far and how they will be managed
- issues—a strategy for capturing and resolving issues
- evaluation – an approach for measuring results and achieving outcomes; and
- a work plan for the short to medium term, which:
- identifies the components of the project
- shows the main deliverables and milestones for each component
- shows the contribution each component makes to the overall project outcomes; and
- contains estimates of resource requirements.
These documents, and any other information the Assurance Review Team deems relevant, will be required prior to the onsite review.
Key focus areas for Gate 0 reviews
The following topics would commonly be considered during a Gate 0 – Business need review. Review teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Review teams may determine that additional topics are critical to the assessment of the project.
1. Policy and context
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Is the project innovative in planning to succeed? |
|
1.2 |
Is there a clear understanding of the required outcomes of the project and are they soundly based? |
|
1.3 |
Does the project demonstrate a clear link with wider government objectives? |
|
1.4 |
Is the business strategy to which this project contributes robust? |
|
1.5 |
Does the project, program or policy require new governance arrangements e.g. cross-portfolios |
|
1.6 |
Does the project align with government policy and initiatives, the broad environment of the entity and the entity’s strategic objectives? |
|
1.7 |
Is the governance framework fit for the purpose? Is there commitment to key roles and responsibilities for this project? |
|
1.8 | Are the required skills and capabilities for this project available, taking into account the entity’s current commitments and capacity to deliver? |
|
1.9 | Is the entity able to learn from experience with this project and other projects? |
|
1.10 | Does this project have a risk management framework? | Defined roles, responsibilities and processes for managing risk across the entity and within the project, with clearly defined routes for escalating risk concerns to senior management. |
2. Business case: scope and stakeholders
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Is the scope of the project understood? |
|
2.2 |
What is the full extent of the project envisaged and why? |
|
2.3 |
What will constitute success? |
|
2.4 |
Who are the stakeholders and do they support the project? |
|
2.5 |
Is the proposed project affordable? |
|
2.6 |
What assumptions is the project based on? What are the major constraints for the project? |
|
2.7 |
Has the supplier market been considered? |
|
2.8 |
Have project controls been determined, especially where constituent projects will be connected with other entities, jurisdictions or the private sector? |
|
2.9 |
What else could affect success? |
|
3. Review of current phase
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
Does the project need to comply with broader government or departmental timing requirements? |
|
3.2 |
Is the project on track? |
|
3.3 |
Have options for potential ways forward been identified? |
|
4. Management of intended outcomes
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Have the required outcomes from the project been identified? |
|
4.2 |
Are the planned outcomes still achievable, or have any changes in scope, relationship or value been properly agreed, and has the business case been reviewed? Is the project time critical? |
|
4.3 |
Are key stakeholders confident outcomes will be achieved when expected? Is it on track to deliver? |
|
4.4 | Is there a plan for achieving the required outcomes? |
|
5. Risk management
Areas to probe |
Evidence expected |
|
---|---|---|
5.1 |
Have the major risks been identified? |
|
5.2 |
How will risks be managed? |
|
5.3 |
Have assurance measures for the project been put in place? |
|
5.4 |
Is there a contingency plan and business continuity plans?
|
|
5.5 |
Have lessons from similar projects been considered? |
|
6. Readiness for next phase (business case)
Areas to probe |
Evidence expected |
|
---|---|---|
6.1 |
Affordability: Are the funds to reach the next phase of the project available? |
|
6.2 |
Are the required internal/external resources available, suitably skilled, and committed to undertake the work? |
|
6.3 |
Achievability: Are the plans for the ‘business case’ phase realistic? |
|
6.4 |
Are appropriate management controls in place? |
|
Where a Gate 0 review is undertaken at a stage in the project later than the start-up stage, the areas to consider in the topic ‘Readiness for Next Phase’ are likely to vary from those listed above. Suggested areas to consider are provided in the following table.
The reason for undertaking a Gate 0 review will be dictated by the circumstances of the particular project, more so than for other reviews. Consequently, the review approach is likely to require more development by the Assurance Review Team than at other Gates.
7. Readiness for next phase (post-business case and beyond)
Areas to probe |
Evidence expected |
|
---|---|---|
7.1 |
Is there a continuing need for the project? |
|
7.2 |
What assumptions have been made about the project? |
|
7.3 |
How will change be managed? |
|
7.4 |
Affordability: Are the funds to reach the next phase available? |
|
7.5 |
Are the required internal/external (individuals and organisations) suitably skilled, available and committed to carrying out the work? |
|
7.6 |
Achievability: Are the plans for the next phase realistic? |
|
7.7 |
Are appropriate management controls in place? |
|
Gate 1 – Business case review
Purpose
At the commencement of a project, an entity will develop a document (typically a business case) that articulates the impetus and business need for the project, together with an assessment of the project’s likely costs, benefits and potential for success. The Gate 1—Business case review comes after a business case has been prepared.
The Gate 1—Business case review focuses on the robustness of a project’s business case. It provides assurance to the entity, through the SRO, that the proposed approach:
- has been adequately researched
- can be delivered within the allocated time; and
- can be delivered with the proposed resources.
At this Gate, understanding and articulating expected benefits guides options analysis and potential delivery approaches. This approach helps increase confidence in decision makers. It also allows a risk assessment to be undertaken against options and their likelihood to achieve the pre-defined benefits.
Review Team expectations in Gate 1 reviews
- Confirming that the business case is robust (that is, it meets the business need, is affordable and achievable, with appropriate options explored and is likely to achieve value for money)
- confirming that potential options for the delivery of desired outcomes have been identified and analysed and appropriate expert advice has been obtained as necessary
- establishing that the feasibility study or assessment has been completed satisfactorily and a preferred way forward has been determined
- confirming that the market’s likely interest has been assessed
- ensuring there is internal and external authority, if required, and that support exists for the project
- ensuring major risks have been identified and a risk management plan has been developed
- confirming quality and benefits management plans are in place, including key performance indicators for the project and its outcomes
- establishing that the project is likely to deliver its business goals and that it supports wider business change where applicable
- confirming that the impact of business process changes on internal and external stakeholders have been assessed
- confirming that the scope and requirements are realistic, clear and unambiguous
- ensuring that the full scale, intended outcomes, timeframes and effect of relevant external issues have been assessed
- ensuring plans exist for the next phase
- validating assumptions in the plan to confirm the project team can deliver the next stage
- confirming that overarching and internal business and technical strategies have been assessed; and
- establishing that quality assurance strategies/plans for the project and its products are in place.
Documents required in Gate 1 reviews
In order to undertake a Gate 1 review, the Assurance Review Team may require access to the following documentation:
- the project brief or mandate with the project’s scope and an explanation of the need for the project
- a project initiation document or equivalent
- details of the project approach, including how to deliver the intended outcome
- a strategy outlining the approach to business change arising from the project (including staff training, new facilities etc. as appropriate)
- an initial assessment of the current and proposed physical and technical environment (e.g. IT infrastructure, workspace facilities)
- a report on the project’s expenditure to date versus its forecasted budget (e.g., cost of scoping or feasibility study versus budget)
- a high-level definition of the business requirements and total scope of the project
- a description of how the project’s success will be evaluated
- a business case addressing business need, affordability, achievability, value for money and range of options estimating the project’s costs and benefits, including some form of feasibility study, sensitivity analysis and market research
- a communications strategy to keep stakeholders informed of the project’s progress
- a quality assurance management strategy
- a list of the major risks, with draft plans for managing them
- a high-level activity, time and resource plan for the whole project
- plans to move the project through to the next stage, Gate 2—Delivery strategy
- how performance is to be reported and monitored
- project organisation—key roles and governance/reporting arrangements
- business impacts, for IT-enabled projects
- estimates and source of funds to cover all work to Gate 2—Delivery strategy
- the authority and approval to proceed
- a benefits management plan; and
- benefits profiles.
These documents, and any other information the Assurance Review Team deems relevant, will be required prior to the onsite review.
Key focus areas for Gate 1 reviews
The following topics would commonly be considered during a Gate 1 – Business case review. Assurance Review Teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Assurance Review Teams may determine that additional topics are critical to the assessment of the project.
1. Business case
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Is there a clear and agreed understanding of the business objectives and how they will be delivered? |
|
1.2 |
Is the impetus for change described in the business case? |
|
1.3 |
Are all relevant government initiatives being addressed? |
|
1.4 |
Is the business case sufficiently linked to the entity outcomes and project with the investment objectives? |
|
1.5 |
Have internal and external factors affecting the project been identified and assessed? |
|
1.6 |
What factors are critical to ensure success? |
|
1.7 |
Can the critical success factors be quantified or measured? |
|
1.8 |
For IT projects, will the business case make a significant contribution to entity IT objectives? |
|
1.9 |
Has the business case examined a wide range of options that will meet the business need? |
|
1.10 |
Is there a clear ‘best option’, or would several options meet the business need? |
|
1.11 |
If there are several options, how was their robustness tested? |
|
1.12 |
Is the argument for the ‘preferred option’ sound? |
|
1.13 |
Does the preferred option meet wider government and entity objectives, standards and business change program? |
|
1.14 |
Does the business case demonstrate that the opportunities for sharing services across entities have been investigated? |
|
1.15 |
Has contract management been considered? |
|
1.16 |
Were the feasibility study and business case completed within time and cost budgets? |
|
1.17 |
Have assumptions been identified and their validity checked? |
|
1.18 |
Has Finance agreed the costings for the project? |
|
2. Project governance and planning
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Has the project governance been considered and is there an overall project management process? |
|
2.2 |
Has a steering committee, or equivalent, been established to oversee the project? |
|
3. Stakeholders
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
Have all the likely stakeholders been identified and are their interests clearly understood? |
|
3.2 |
Do stakeholders support the preferred option? (This includes the potential or recommended delivery approach and mechanisms.) |
|
3.3 |
Are stakeholder issues being addressed? These include: • communications • public relations • environmental issues • personnel • statutory processes |
|
4. Risk management
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Are there processes in place to identify, assess and monitor current, anticipated and emerging risks? |
|
4.2 |
Have any issues raised in a previous review been resolved satisfactorily? |
|
4.3 |
Have the risks for each of the options been evaluated? |
|
4.4 |
Have the risks for the preferred option been fully assessed? |
|
4.5 |
Have the ‘worst case’ costs associated with risks been assessed? |
|
4.6 |
Are risk management costs and time implications included in the cost and time estimate or are they treated as a contingency? |
|
4.7 |
Does the project break new ground? |
|
4.8 |
Would it be better to break the project down into a series of small steps? |
|
5. Readiness for next phase (delivery strategy)
Areas to probe |
Evidence expected |
|
---|---|---|
5.1 |
Is there an overall project structure for the delivery stage? |
|
5.2 |
Is there a realistic plan to reach the next stage of the project (Gate 2— Delivery strategy)? |
|
5.3 |
Have any requirements for external specialist advice been determined? Are the necessary internal and external skills available at the right time and in the right numbers? Is there a training need, and if so, what is it? |
|
5.4 |
Is the project timetable realistic? Does it take into account any statutory lead times and time required for approvals? |
|
5.5 |
Is there a clearly defined project organisation for the next stage of the project, with agreed roles and responsibilities?
|
|
5.6 |
Are the necessary funds to reach Gate 2—Delivery strategy identified? |
|
Gate 2 – Delivery strategy review
Purpose
Following the Gate 1—Business case review, the project governance board or steering committee and/or SRO will have determined whether the project is feasible and has a robust high-level business case.
The Gate 2 review focuses on evaluating the delivery strategy to provide assurance to the project steering committee and/or SRO that the selected delivery approach is appropriate and that it:
- establishes a clear definition of the project
- establishes a plan for its implementation; and
- has made an assessment of the project’s potential for success.
Where it includes an acquisition strategy it also provides assurance that the project is ready to invite proposals or tenders from the market.
During the delivery stage of a project, a clear understanding of the expected benefits helps focus on the delivery of the activities that will achieve the expected benefits. Some interim benefits may be achieved during this stage, which provide an excellent indicator for the ongoing viability of the project.
Review Team expectations in Gate 2 reviews
- Confirming the business case, now that the project is fully defined
- ensuring that the project’s plan is appropriately detailed and realistic, through to completion, and includes a change management strategy
- ensuring that the project’s controls and organisation are defined (financial controls are in place and the resources are available)
- confirming funding availability for the whole project
- confirming that the development and delivery approach and mechanisms are still appropriate and manageable
- ensuring that the procurement strategy is robust and appropriate and provides an exit strategy, if necessary
- checking that the supplier market capability and track record (or existing supplier’s capability and performance) are fully understood
- confirming that the procurement approach will facilitate good client/supplier relationships
- confirming that the procurement plan facilitates an outcome providing value for money
- confirming that appropriate project performance measures and tools are being used
- confirming that quality assurance procedures have been applied consistently with the previous review
- ensuring that delivery outputs are being mapped to expected benefits
- confirming compliance with IT infrastructure and security requirements, as appropriate for IT projects; and
- confirming compliance with building codes, occupational health and safety and sustainability requirements, as appropriate for construction projects.
Documents required in Gate 2 reviews
In order to undertake a Gate 2 review, the Assurance Review Team may require access to the following documentation:
- an updated business case containing a plan for realising benefits
- a report on the project’s expenditure to date versus its forecasted budget, where appropriate
- a plan for managing the business change
- specifications of the project’s expected outputs and outcomes
- the procurement strategy and justification for the approach
- draft tender documents, including the evaluation plan and draft contract
- a description of the high-level requirements of the project
- the proposed implementation strategy for implementing the new service/works contract
- a current risk management plan, risk register and issues register
- current and planned business/technical policies, strategies and constraints (e.g., occupational health and safety standards)
- an outline of project plans to completion, and detailed plans for the next stage
- the results of any business, commercial or technical benchmarking
- market intelligence and research material
- the current communications strategy and plan
- project quality documentation
- a strategy for measuring project performance, including occupational health and safety, as appropriate for construction projects
- benefits management strategy
- benefits profiles; and
- benefits realisation plan.
These documents, and any other information the Assurance Review Team deems relevant, will be required prior to the onsite review.
Key focus areas for Gate 2 reviews
The following topics would commonly be considered during a Gate 2 – Delivery strategy review. Assurance Review Teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Assurance Review Teams may determine that additional topics are critical to the assessment of the project.
1. Business case and stakeholders
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Strategic fit: Does the business case continue to demonstrate the business need and contribute to the business strategy? |
|
1.2 |
Options explored: Is the preferred way forward still appropriate? |
|
1.3 |
Value for money: Is the proposed commercial arrangement likely to achieve value for money? |
|
1.4 |
Affordability: Are the costs within current budgets? Is the project’s funding affordable and supported by key stakeholders? |
|
1.5 |
Achievability: Is the entity still realistic about its ability to achieve a successful outcome? |
|
1.6 |
Is there a clear definition of the total project scope? |
|
1.7 |
Are the issues relating to business change understood? Is there an initial plan to address these issues? |
|
1.8 |
Is the entity fully committed to the project? Do stakeholders support the project? |
|
1.9 |
Are the benefits to be delivered by the project understood and agreed to with stakeholders? Is there an initial plan for realising benefits? |
|
2. Delivery and procurement approach
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Will the project be attractive to the market? |
|
2.2 |
Are the business objectives clearly understood by the entity and likely to be understood by suppliers? |
|
2.3 |
Are the project outputs/outcomes accurately reflected in the requirement specification? |
|
2.4 |
Have options for the proposed procurement approach been evaluated, including sources of supply? |
|
2.5 |
Is the procurement strategy consistent with the approved business case? |
|
2.6 |
Is the selected procurement strategy defined and endorsed? |
|
2.7 |
Have factors that may affect the procurement strategy been addressed? |
|
2.8 |
Will the procurement strategy facilitate communication and co-operation between potential suppliers and the client? |
|
2.9 |
Has the proposed procurement procedure been evaluated? |
|
2.10 |
Is there adequate knowledge of existing and potential suppliers? |
|
2.11 |
Is the contract management strategy robust? |
|
2.12 |
Has the project team complied with the relevant procurement policies and guidelines in preparation of the tender documents? |
|
2.13 |
Is the evaluation strategy (including how to demonstrate value for money) accepted by stakeholders and compliant with the Commonwealth Procurement Rules? |
|
3. Review of current phase
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
Is the project being managed in accordance with its governance framework, stakeholder engagement plan, project plan and other critical documents? |
|
3.2 |
What caused any variations such as over-runs or under-runs? |
|
3.3 |
What actions are necessary to prevent variations recurring in other phases? |
|
3.4 |
Were there any assumptions made at Gate 1, or earlier reviews that have not been verified? |
|
4. Risk management
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Are the major risks identified, understood, financially evaluated and considered in determining the procurement strategy? |
|
4.2 |
Are there risk management plans in place? |
|
4.3 |
Have all issues raised in previous review(s), and during the course of the project, been satisfactorily resolved? |
|
4.4 |
Are the following external issues being addressed:
|
|
5. Readiness for next phase (investment decision)
Areas to probe |
Evidence expected |
|
---|---|---|
5.1 |
Is the project plan for the remaining phases realistic? |
|
5.2 |
Are the project’s timelines reasonable and compliant with relevant procurement policies and guidelines? |
|
5.3 |
What are the arrangements for the next stage of the project? Have its activities been defined and resourced? |
|
5.4 |
Does the project have resources with the appropriate skills and experience where required? |
|
Gate 3 – Investment decision review
Purpose
The Gate 2—Delivery strategy review considered the procurement strategy prior to inviting proposals or tenders against the fully developed requirements specification. Prior to the Gate 3—Investment decision review, in most cases potential suppliers and partners will have submitted their proposals or tenders, and the evaluation panel will have analysed these and recommended the proposal that offered the best value for money.
The Gate 3—Investment decision review is usually conducted before taking the procurement action recommended by the evaluation panel. This review assesses the appropriateness of the supplier selection process, how business needs are being met and the processes in place for contract delivery. It also determines:
- how well the process has been managed
- the approach undertaken to determining value for money is consistent with the procurement plan criteria for doing so
- how both the entity and the supplier can implement and manage the proposed solution; and
- that processes are in place to achieve a successful outcome after contract award (or equivalent).
Review Team expectations in Gate 3 reviews
- Confirming the business case and benefits realisation plan, now that firm proposals have been received
- checking that all the policy, statutory and procedural requirements were followed throughout the procurement process
- confirming that the recommended contract decision is likely to deliver the specified outputs/ outcomes on time, within budget and that value for money has been adequately assessed
- confirming the existence of an exit strategy and arrangements for retendering
- ensuring that management controls are in place to manage the project through to completion
- ensuring there is continuing support for the project
- confirming that the approved procurement strategy has been followed
- confirming that the development and implementation plans of both the entity and the supplier/ partner are sound and achievable
- checking that the entity has prepared for developing new business processes, where needed, and for implementing and operating new services or facilities, as well as the transition process
- confirming that there are plans for risk management, issue management, quality management, key benefits management and change management (technical and business) and that these plans are shared with suppliers
- confirming that technical implications of a project have been addressed; and
- confirming that the proposed procurement is within the approved financial limits.
Documents required in Gate 3 reviews
In order to undertake a Gate 3 review, the Assurance Review Team may require access to the following documentation:
- project management information, including:
- strategies for managing the risks and issues, including plans and risk log showing that risks are being identified and managed
- the plans for implementing business process change and handling changes in requirements
- the quality management plan describing the how the quality of project deliverables will be assured
- the service management arrangements defining service management responsibilities for the entity and supplier, service delivery procedures and supplier performance is to be measured
- the benefits management strategy, plans and responsibilities for delivery
- the procurement strategy
- any operational requirements; and
- the draft contract.
- confirmation that the procurement strategy approved at the Gate 2 review has been followed
- confirmation that the negotiated and agreed solution(s) remain within the original criteria
- plans and assessments of probity throughout the procurement process
- adequate plans from the supplier for development and implementation
- an evaluation report containing recommendation and justification of the selected supplier and plans for debrief of unsuccessful suppliers
- draft project plans through to completion and detailed plans for the next stage
- a project timetable proposed by suppliers as part of their tender
- an updated communications plan
- confirmation of adequate financing and budget, and authority to proceed
- benefits management strategy
- benefits profiles; and
- benefits realisation plan.
These documents, and any other information the Assurance Review Team deems relevant, will be required prior to the onsite review.
Key focus areas for Gate 3 reviews
The following topics would commonly be considered during a Gate 3 – Investment decision review. Assurance Review Teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Assurance Review Teams may determine that additional topics are critical to the assessment of the project.
1. Business case and stakeholders
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Is the project still required? |
|
1.2 |
Are the requirements of the business case complete or does the tender decision require the business case to be amended and/or re-assessed? |
|
1.3 |
Does the recommended way forward meet the business need? |
|
1.4 |
Has the most appropriate option been selected? |
|
1.5 |
Does the commercial arrangement represent value for money, with an appropriate level of quality over the whole life of the project? |
|
1.6 |
Is the entity realistic about its ability to manage the change? |
|
1.7 |
Does the business case still demonstrate affordability when incorporating the proposal? |
|
1.8 |
Is there an agreed plan for the realisation of anticipated benefits? |
|
1.9 |
Have suitable stakeholders been involved and, if so, what has been their involvement? |
|
2. Assessment of the proposed solution
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Does the proposed solution still meet the business needs and government and entity objectives? |
|
2.2 |
Have the suppliers proposed any alternatives or other options in addition to a fully compliant bid? |
|
2.3 |
Will the proposed bid deliver the business need described in the business case? |
|
2.4 |
Has the proposed solution affected the strategy for business change? |
|
2.5 |
Has the proposed solution affected the expectations of business benefits? |
|
2.6 |
Are the entity and supplier prepared for the development (where there are new systems and processes), implementation, transition and operation of any new services? |
|
2.7 |
Are there plans and processes to address future issues, both business and technical? |
|
2.8 |
Is there clear allocation and understanding of responsibilities between all parties, in addition to any contractual obligations? |
|
2.9 |
Are resources available for the supplier to fulfil its obligations within the contract? |
|
2.10 |
Have the technical implications been assessed? |
|
2.11 |
Does the project have resources with the appropriate skills and experience to achieve the intended outcomes of the investment? |
|
2.12 |
Is the proposed procurement within financial approvals and is there adequate budget to accommodate the procurement? |
|
3. Review of current phase
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
Is the project under control? (i.e. running to plan and budget) |
|
3.2 |
What caused any variations? |
|
3.3 |
What actions are necessary to prevent variations recurring in other phases? |
|
3.4 |
Have all the assumptions from Gate 0, 1 and 2 reviews been validated? |
|
3.5 |
Have all the required entity procurement and technical checks been undertaken? |
|
3.6 |
Did the project team follow the planned steps in the procurement strategy? |
|
3.7 |
Has the procurement process adequately managed probity? |
|
4. Risk management
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Are risk and issue management plans up to date? Are they being monitored? |
|
4.2 |
Have all major risks that arose during this stage been resolved? |
|
4.3 |
Are there arrangements to minimise risks to the business in the event of major problems during implementation and rollout? |
|
5. Readiness for next phase (readiness for service)
Areas to probe |
Evidence expected |
|
---|---|---|
5.1 |
Is the working relationship likely to succeed? |
|
5.2 |
Are all resources and internal funds in place? |
|
5.3 |
Are the supplier’s project, risk and management plans adequate and realistic? |
|
5.4 | Does the entity’s plan reflect the supplier’s plans? |
|
5.5 | Are the long-term contract, administration plan and performance measurement process complete? |
|
5.6 | Are all mechanisms and processes for the next phase in place? | The project plan confirms arrangements for management, monitoring, transition and implementation. |
5.7 | Are the service management plan, administration and service level arrangements complete? |
|
5.8 | Is the management process for service change complete? |
|
5.9 | Is there an acceptance strategy or commissioning strategy, as applicable? |
|
5.10 | Is there an implementation strategy? |
|
Gate 4 – Readiness for service review
Purpose
The Gate 3 – Investment decision review covered the activity leading up to contract, including the assessment of the process for arriving at the procurement decision prior to commitment. Once the investment decision has been executed, the project can be delivered in accordance with the business requirement specifications.
The Gate 4—Readiness for service review determines:
- that the solution is robust before delivery
- how ready the entity is to implement the business changes that occur before and after delivery; and
- that there is a basis for evaluating ongoing performance.
For infrastructure projects, this review takes place as soon as practical (preferably before commissioning), to take advantage of the availability of the procurement team. For IT-supported business change, this review takes place after all testing has been completed, including business integration and assurance testing, and before roll-out or release into production.
As the outputs are ready to be embedded into the entity, business change effort ramps up to ensure that the outputs can be effectively implemented, with as little impact on current operations as possible. A benefits focus ensures that business change can be linked directly to achieving the stated benefits. Not only does it help drive the change effort, measuring benefits before and after business change also shows the effectiveness of the change.
Review Team expectations in Gate 4 reviews
- Checking that the contractual arrangements are up to date, and that the current phase of the contract is properly completed and documented
- confirming that there are plans for managing the working relationship with the supplier, with agreement at appropriate levels
- confirming that all parties have agreed plans for managing risks
- confirming key benefits documentation has been aligned to delivery
- checking that risks and issues are being managed effectively and do not threaten implementation, and evaluate the risk of proceeding with the implementation, if there are any unresolved issues
- confirming that commissioning plans have been developed and that they identify and address key areas such as business integration, change management and business continuity management. Entities are encouraged to refer to internal policies and industry best practice in preparing these plans
- checking that the commissioning plans have been assessed for risks, affordability and robustness
- confirming that the business process changes are being implemented
- confirming that entity and supplier implementation plans are still achievable
- confirming the business has the necessary resources and is ready to implement the services and the business process change(s)
- confirming that there are management and organisational controls to manage the project through implementation and operation
- confirming that all parties have agreed plans for training, communication, roll-out, production, release and support
- confirming that any required testing (e.g. commissioning of buildings, business integration and user acceptance testing) is done to the end-user’s satisfaction and that the project’s Senior Responsible Official is ready to approve implementation
- checking that there are feasible and tested contingency and exit arrangements
- confirming that defects or incomplete works are identified and recorded
- checking that the business case is still valid and unaffected by internal and external events or changes, and that the original projected business benefit is likely to be achieved
- checking that lessons for future projects are identified, recorded and are proposed for dissemination; and
- ensuring processes and procedures are in place to ensure the long-term success of the project.
Documents required in Gate 4 reviews
In order to undertake a Gate 4 review, the Assurance Review Team may require access to the following documentation:
- an updated requirements definition, with any changes agreed during the period up to Gate 3 review
- an updated business case and plans for benefits realisation which reflect the effect of any requirements changes, and the plans for service delivery
- a plan for performance measurement
- a risk management plan
- a benefits management plan
- close-out documentation (if the project ends at implementation and a new one begins) and status reports and reconciliations for
- cost versus budget
- actual versus planned schedule
- risk management plans
- communication and external relations plans
- environmental performance documentation
- adherence to statutory requirements
- the updated contract
- an assessment of contractual issues during the project to date
- details of any facilities not provided to the required specification and any missing or deficient items, with agreed plans for addressing any outstanding issues
- draft project plans through to the completion stage and detailed plans for the next stage
- a test plan and test reports
- progress reports on development and construction
- updated risk and issues logs, including residual risks
- updated contingency and reversion plans
- the plan for managing change, including expected changes to requirements over time
- an updated occupational health and safety file, for construction projects
- records of building site visits
- updated contingency and reversion plan and information assurance documentation (accreditation), for IT-enabled projects
- lessons learnt during the project (if the project ends at implementation)
- operational and maintenance instructions and warranties
- capability assessment for operating and maintaining the solution
- operational governance models
- deployment/transition plan
- implementation strategy and plan
- operational resource plan
- operational plans:
- business/technical support models,
- business continuity and disaster recovery, and
- back-up, restore, archive and retention procedures.
- data management, and data quality plan
- architecture and design specs – particularly those relating to the following and how they will be maintained, monitored and reported:
- specifications for externally available interfaces
- availability requirements
- performance requirements
- capacity requirements.
- security threat and risk assessment
- security support model – covering monitoring, reporting, denial of service attacks, user access controls
- application software maintenance plan, strategy, manuals covering installation, configuration and maintenance documentation; and
- updated benefits realisation plan.
These documents, and any other information the Assurance Review Team deems relevant, will be required at the planning meeting or during the onsite review.
Key focus areas for Gate 4 reviews
The following topics would commonly be considered during a Gate 4 – Readiness for service review. Assurance Review Teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Assurance Review Teams may determine that additional topics are critical to the assessment of the project.
1. Business case and stakeholders
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Is the project still required? |
|
1.2 |
Does the project still meet the business needs and objectives of the relevant users and stakeholders (i.e. government departments, interest groups etc.)? |
|
1.3 |
Is the business case still valid? |
|
1.4 |
Are there any changes between contract execution and completion of transition/testing stages that affect the business change project? |
|
1.5 |
Is the entity ready for business change? |
|
1.6 |
Can the organisation implement the new services whilst maintaining existing services? |
|
1.7 |
Are there appropriately skilled and experienced staff available? |
|
2. Review of current phase
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Do the products and/or services delivered meet the acceptance criteria? |
|
2.2 |
Is the project under control (i.e. running to plan and budget)? |
|
2.3 |
Have all the stakeholder issues been addressed? |
|
2.4 |
Are all testing and commissioning/ acceptance (or transition) procedures complete? |
|
2.5 |
Have all parties accepted the commissioning/test results and any required action plans? |
|
2.6 |
Are there workable and tested contingency and back-out plans for roll-out, implementation and operation? |
|
2.7 |
Have the supplier and all internal and external parties agreed on the implementation plans? These could include: • management of change • migration and data transfer • client and supplier implementation • roll-out • post-implementation review. |
|
2.8 |
Have any changes to the contract been previously forecast, accurately recorded and approved? |
|
2.9 |
Is there a training plan and curriculum? |
|
3. Risk management
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
Have the risks and issues that arose in the awarding of the contract award and implementation phase been properly managed? |
|
3.2 |
If there are unresolved issues, what are the risks of implementing rather than delaying? |
|
4. Readiness for next phase (benefits realisations)
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Are all project elements ready for service? |
|
4.2 |
Is the entity ready to adopt new ways of working, where applicable? |
|
4.3 |
Is the long-term contract management process in place? |
|
4.4 |
Is there a process to manage and measure performance? |
|
4.5 |
Is there a process to manage measure and report benefits? |
|
4.6 |
Have ongoing operation and maintenance been considered in detail? |
|
4.7 |
Is there a process for post-implementation reviews? |
|
Gate 5 – Benefits realisation review
Purpose
The Gate 4—Readiness for service review assessed whether the solution was robust before delivery, and that the entity was ready for the implementation and had a basis for evaluating ongoing performance.
A Gate 5—Benefits realisation review is not a post-implementation review. It takes place after the entity has carried out a post-implementation review or similar major review. It makes use of findings from that internal review, together with an assessment of organisational learning, as evidence of good practice but may or may not include a full review of plans for the future.
The Gate 5—Benefits realisation review focuses on ensuring that the project delivered the benefits and value for money identified in the business case and benefits realisation plans. A Gate 5 review is generally held six to twelve months after commissioning of the product(s) or introduction of the service, when evidence of the benefits is available.
The scope and frequency of Gate 5 reviews will vary depending on the project and contract characteristics. A single Gate 5 review is generally enough for most projects, however, the Assurance Review Team will make this determination.
Review Team expectations in Gate 5 reviews
- Assessing whether the business case for the project at Gate 3 and at Gate 4 was realistic
- assessing whether the anticipated benefits are being delivered
- confirming that the responsible entity or supplier continue to have the necessary resources to successfully manage the contract
- confirming continuity of key personnel involved in contract management roles
- assessing the ongoing requirement for the contract to meet the business need. Ensure that if circumstances have changed, the service delivery and contract are adapting to the new situation. Changing circumstances could affect partner, relationship, service, change, contract, benefits and performance management
- where changes have been agreed, ensuring they do not compromise the original contract
- ensuring there is ongoing contract development to improve value for money
- assessing the application of the contract management procedures to date to manage the contract
- confirming that there are plans to manage the contract to its conclusion
- assessing lessons learnt and methodology for sharing these with peers within the NCE and across government; and
- confirming the validity of the exit strategy and arrangements for retendering, where applicable.
Documents required in Gate 5 reviews
In order to undertake a Gate 5 review, the Assurance Review Team may require access to the following documentation:
- an updated business case that reflects actual operating conditions, compared with the business case reviewed at the Gate 4—Readiness for service review
- report on the findings of any post-implementation reviews (or equivalent major post-project reviews)
- an assessment of the benefits delivered to date and expectations for the future
- a summary of contract changes since the Gate 4 review
- plans for contract and service improvement
- resources, skills appraisals and personnel plans to continue managing the contract
- reports on stakeholder issues
- plans for disposal of any assets at the end of the contract (e.g. resources, buildings, staff, intellectual property rights)
- adherence to building code, occupational health and safety, and sustainability requirements, for construction projects
- adherence to security documents, for IT-enabled projects
- contract management reports; and
- project records regarding change and dispute management.
Key focus areas for Gate 5 reviews
The following topics would commonly be considered during a Gate 5 – Benefits realisation review. Review teams are expected to use their own expertise in determining the relevance and appropriateness of these topics for the specific project under review. Review teams may determine that additional topics are critical to the assessment of the project.
1. Business case and benefits management
Areas to probe |
Evidence expected |
|
---|---|---|
1.1 |
Is the business case still valid? |
|
1.2 |
Have the business benefits been realised as set out in the business case? Did the entity achieve other benefits? |
|
1.3 |
Have the needs of the business and/ or end-users changed? |
|
1.4 |
Have all the governance and stakeholder issues been addressed? These include:
|
|
1.5 |
Are the users satisfied with the operational service? |
|
2. Review of operating phase
Areas to probe |
Evidence expected |
|
---|---|---|
2.1 |
Is the service/facility operating to defined parameters? |
|
2.2 |
Have the project documentation, training material and training program been delivered and kept up to date? |
|
2.3 |
Are the contractual relationships satisfactory? |
|
2.4 |
Are there plans for continued contract management? |
|
2.5 |
Are plans for ongoing risk management up to date? |
|
2.6 |
Have the contract management procedures been successful to date? |
|
3. Plans for ongoing improvements in value for money
Areas to probe |
Evidence expected |
|
---|---|---|
3.1 |
What is the scope for improved value for money? Can more be done for less? Could the provider deliver better service quality at the same price? Can maintenance costs be reduced? |
|
3.2 |
Has the entity benchmarked its contract-related processes by comparing them with other equivalent organisations involved in similar relationships? |
|
3.3 |
Are commercial mechanisms providing appropriate incentives? |
|
4. Plans for ongoing improvements in performance
Areas to probe |
Evidence expected |
|
---|---|---|
4.1 |
Is the entity setting realistic targets for continuous improvement year-on-year from this service? |
|
4.2 |
Is the entity tracking its progression to improved performance and the flow of results through key milestones and the business planning cycle? |
|
4.3 |
Does the entity have performance measures to cover all aspects of the contract? |
|
4.4 |
Do the performance measures selected offer clear and demonstrable evidence of the success (or otherwise) of the contract? |
|
4.5 |
Are performance measures related to delivery or capability improvement tracked against an existing baseline? |
|
4.6 |
Are there performance assessment measures for:
What does the entity want to have achieved by the end of the contract period? |
|
5. Readiness for the future
Areas to probe |
Evidence expected |
|
---|---|---|
5.1 |
Is there an ongoing need for the service? |
|
5.2 |
If the service will be needed in the future, what is its likely scope? |
|
5.3 |
Are there any major issues with the current contract that could affect the approach to retendering the service? Factors to consider include:
|
|
6. Review of organisational learning and maturity targets
Areas to probe |
Evidence expected |
|
---|---|---|
6.1 |
Does the entity have a well-defined and effective process implemented for identifying lessons learnt from the project, embedding them internally and sharing them across government? |
|
6.2 |
Has there been a review of how well the project was managed? |
|
6.3 |
Are suppliers encouraged to learn from experience? |
|
Gateway for programs
Purpose
The primary purposes of a Gateway program review are to review the outcomes and objectives for the program and confirm that they make the necessary contribution to the overall strategy of the organisation and its senior management; and align effectively with broader high level government policy objectives and initiatives.
First stage program reviews
First stage program reviews are conducted before government approval to ensure that any pertinent outcome/s can be addressed in time to contribute to the government’s deliberations. Where this is not practical, (e.g. rapid/urgent decisions) the review can be conducted as soon as possible after the government’s approval, but before substantial program design, definition and implementation. This review would also help to define the program by examining the business need and business case.
These reviews will assess whether stakeholders’ expectations of the program are realistic, in relation to outcomes and benefits, resource requirements and capability, and timetable and achievability.
A program review conducted at the start up or Budget pre-decision stage is particularly valuable as it helps to confirm the way forward is achievable before implementation plans have been finalised and major investment occurs. These reviews will assess whether stakeholders’ expectations of the program are realistic, by reference to outcomes and benefits, resource requirements, capability, timetable, and achievability.
This type of review may assist entities in defining the program by examining the business need and formulation of the business case and can be conducted whenever the priority or the scope of the program changes significantly.
Mid stage program reviews
Mid-stage program reviews will assess the program execution with the number of these reviews being determined by the complexity, timeframe and risks attached to the program.
These reviews may be conducted multiple times and on a regular basis (generally at intervals of no more than 12-18 months), depending on the outcomes of previous reviews and/or where there is a lengthy period between decision-points, staged implementation or an opportunity to assess the programs’ maximisation of benefits.
The scope of each review is determined through discussions between the entity, the Assurance Review Team and Finance.
End stage program reviews
End stage program reviews will focus on the realisation of benefits. The review will confirm links to the business case are still robust and that senior management support clarity of understanding about the required outcomes. The review also confirms that expected outcomes are being achieved against the entity’s performance indicators and targets (i.e. as set out in the Portfolio Budget Statement) and that no outstanding issues remain.
These reviews focus on program closure including program controls, records management and the identification and application of lessons learned as well as the delivery of the intended outcomes and benefits.
Key focus areas for program reviews
This section outlines the six key focus areas, the context and scope for each review stage, and guidance on areas to probe, including examples of the types of evidence expected. Note that the six key focus areas remain constant between the three stages (First, Mid and End), with the emphasis for the review reflecting the stage of development for the program.
Recognising that each program is unique and that as circumstances change, the areas to probe tabled below are designed to be used as a guide rather than a complete checklist of mandatory items.
Review teams are expected to apply their own expertise to determine the relevance and appropriateness of each topic for each review stage.
1. Policy context and strategic fit
How to use this section for: |
||
First stage program review |
When this review is conducted very early in the program lifecycle, information may be uncertain as options are being explored. Plans for achieving outcomes are likely to be unclear; however, there needs to be evidence of high-level plans for the way forward or a set of options for consideration, with a preferred option identified and a reasonably clear indication of critical success factors and how they will be measured. Strategic fit: there must be a demonstrable link to the entity corporate strategy. For instance, why is this program needed? Governance: the governance framework will be in outline, but a clear owner for the program. Delivery: capability to deliver will be considered at a high level, ideally supported by estimates based on evidence from similar initiatives. Lessons learned: mechanisms put in place to learn lessons regardless of the stage in the program lifecycle. Risks: major risks identified at a high level, even at this early stage, with an indication of how they will be managed, including requirements for contingency plans. Implementation risks that may undermine the timely and effective achievement of intended outcome as is advice on mitigation strategies to reduce these risks. At program initiation, all areas in this section will need thorough investigation as they provide the foundation for successful delivery. |
|
Mid stage program review |
The focus at this stage is whether assumptions or circumstances have changed, e.g. a change in policy direction, continued availability of skilled resources. |
|
End stage program review |
The critical area at the final-stage review is to confirm the link to business strategy is still robust and supported by senior management. |
|
Areas to probe |
Evidence expected |
|
1.1 |
Is there a clear understanding of the required outcomes and are they soundly based? |
|
1.2 |
Does the program break new policy ground? |
|
1.3 |
Does the program sponsor and governance group agree with the business strategy and is the strategy robust? |
|
1.4 |
Does the program demonstrate a clear link with wider government and the NCE’s objectives – does it reflect the current business policy and environment and is it aligned with the business strategy? |
|
1.5 |
Is there a continuing need for the program? |
|
1.6 |
Have other delivery options been considered? |
|
1.7 |
Are there any strategic risks arising from the implementation of this program? |
|
1.8 |
Does the program involve other entities or portfolios? |
|
1.9 |
Has the entity managed similar programs? |
|
1.10 |
Are the key program assumptions explicit and are they still valid? |
|
1.11 |
Are the program’s milestones defined by legislation or by government commitment? |
|
2. Business case and stakeholders
Assess the program’s business case which sets out overall costs and the planned benefits realisation. Ensure the program is supported by a business case and stakeholders, and takes into account the risks, potential change impacts and integration with other internal and external programs.
How to use this section for: |
||
First stage program review |
The level of detail and completeness of the business case will reflect that amount of certainty associated with the program at that point. Initially, the program may tolerate high levels of uncertainty; estimates will be very approximate, with high levels of potential variance. The business case is recommended to be developed in tandem with the program and benefits realisation plans. There must be a clear understanding of the program outcomes from the early stages, even though the overall scope and the way forward may not be clear. The measures of success will be in the program outline. Key stakeholders identified and engaged, especially for those programs, which involve several entities. The components of the program, sub-programs and projects and their resource requirements will not be certain at this stage. Consider early indicators of the additional factors affecting success, which will vary significantly depending on the program. Program controls will not have been established in detail. At program initiation, all areas in this section will require thorough investigation. |
|
Mid stage program review |
These reviews provide an opportunity to revisit program assumptions such as stakeholder support, program affordability and the effectiveness of program controls. It is essential to have effective change control mechanisms in place relating to program scope, interdependencies, master program schedule development and resources (e.g. a change in policy direction, capability and continued availability of skilled resources). Assumptions will need to be revisited, in particular:
|
|
End stage program review |
The main areas to investigate are continued clarity of understanding about the required program outcomes and support for stakeholders as the program closes. Consider program controls and records management. |
|
Areas to probe |
Evidence expected |
|
2.1 |
Is there a robust business case for the program, with links to the individual sub-programs and projects? Is it up to date? |
|
2.2 |
Is the scope of the program clear? Does it overlap or interface with other internal or external policies or programs? |
|
2.3 |
Have the stakeholders been identified and do they support the program? |
|
2.4 |
Does the program:
|
|
2.5 |
Are key stakeholders confident outcomes will be achieved when expected? |
|
2.6 |
Have the program |
|
2.7 |
Have program controls been determined, especially where constituent projects will join other entities? |
|
2.8 |
Has a delivery strategy been developed? |
|
2.9 |
Is there a clear understanding of what constitutes success? |
|
2.10 |
What are the additional factors that could affect success? Are there risks associated with:
|
|
3. Risk management
Review the risk management arrangements for identifying and managing program and project level risks, including external risks such as changing business priorities and competing government priorities.
How to use this section for: |
||
First Stage Program Review |
If the first review is very early, the major risks may have just been identified at a high level with an indication of how they will be managed and initial consideration of the requirements for contingency plans. At program initiation all aspects of risk management must be probed thoroughly. |
|
Mid Stage Program Review |
The focus is checking that risk management remains effective and appropriate. Implementation risks inherent in working with other entities, jurisdictions and contracted service providers adequately identified, assessed and treated. |
|
End Stage Program Review |
The status of the risk register at program closure will be the principal area to investigate and will consider which risks have now been closed/removed and which will be transferred to the risk register for a new initiative or corporate risk log. |
|
Areas to probe |
Evidence expected |
|
3.1 |
Is there a framework for managing issues and risk to this program? |
|
3.2 |
Have the major risks been identified? |
|
3.3 |
Have assurance measures for the program been put in place? |
|
3.4 |
Is there a contingency plan and business continuity plans? |
|
3.5 |
Have lessons from similar programs been considered? |
|
4. Review of current phase
Review the current status of the program in terms of budget, schedule and progress with outcomes, against the performance measures set out in the relevant Portfolio Budget Statements, the approved business case and benefits management plans.
How to use this section for: |
||
First Stage Program Review |
This section would not normally apply but some of the topics may need to be considered. |
|
Mid Stage Program Review |
All areas need to be investigated thoroughly to confirm the program remains on track and issues are being managed effectively. |
|
End Stage Program Review |
Confirm the expected outcomes have been achieved against the performance indicators and targets, as set out in the relevant Portfolio Budget Statements, and the Business Plan and that no outstanding issues remain. |
|
Areas to probe |
Evidence expected |
|
4.1 |
Are the program’s key milestones compliant with broader government or entity timing requirements? |
|
4.2 |
Is the program on track in relation to planning and/or delivery? |
|
4.3 |
Have problems occurred and if so how have they been resolved? |
|
4.4 |
Have options for potential ways forward been identified? |
|
4.5 |
Have lessons learned been shared? |
|
5. Assessment of intended outcomes and benefits
Review the management of intended outcomes and benefits by analysing governance arrangements for leading, managing and monitoring the program, including roles and responsibilities for managing risk, interdependencies between other programs/projects, including those of other entities.
How to use this section for: |
||
First stage program review |
If the first review is very early, the key aspects to investigate thoroughly are:
Plans for achieving the outcomes are likely to be unclear at an early stage. There needs to be evidence of high-level plans for the way forward or a set of options for consideration, with a preferred option identified and a reasonably clear indication of how success will be measured, e.g. a trajectory for take-up of a service. At program initiation, all areas must be investigated thoroughly to confirm expectations for delivery are realistic and performance can be measured with reasonable accuracy. |
|
Mid stage program review |
The Mid-Stage review checks plans to deliver outcomes and ensure benefits remain achievable. It focuses on ensuring that everything is in place to deliver the required outcomes and that information obtained will be acted on to improve the quality of the implementation. At this stage, a benefits realisation strategy and plan needs to clearly show what will happen, where and when the benefits will occur and who will be responsible for their delivery. It is important to be clear about handover and responsibilities for ongoing operations in the changed state (for instance when the benefits will actually be harvested). Integrating this strategy with the overall Program Master Schedule will help to ensure ownership and ongoing management of benefits is maintained after the program has been closed and/or handed over to its business owner. |
|
End stage program review |
The topics in this section would primarily be on benefits realised and lessons learned at program closure. |
|
Areas to probe |
Evidence expected |
|
5.1 |
Have the policy/program outcomes been identified? |
|
5.2 |
Are the planned outcomes achievable, or have any changes in scope, relationship or value been properly agreed? |
|
5.3 |
Is the program on track to deliver? |
|
5.4 |
Is there a plan for monitoring and achieving the required outcomes? |
|
5.5 |
How will change be managed? |
|
5.6 |
Is a benefits management plan active and are benefits being monitored and reported? |
|
5.7 |
Is granting activity part of the program? Do the grants management processes align with the seven key principles prescribed in the Commonwealth Grants Rules and Guidelines?[7] |
|
5.8 |
Where procurement is part of the program, how is capability and capacity for acquisition to be managed? |
|
6. Readiness for next review stage
Assess the entity’s readiness and transition to the next review stage phase by considering how the entity plans to achieve the required outcome, including detailed measures for the management of other entities, jurisdictions or other parties contributing to the outcome.
This section will provide a stocktake of what has been achieved to date and what will be required to move to the next stage. It will assess resources, capabilities and
inter-dependencies.
How to use this section for: |
||
First stage program review |
If the first review is very early, plans may be in too early a stage of development to provide reliable evidence. Where this is the case, this review would examine planned steps to program initiation; all areas would apply to strategic thinking and scope. |
|
Mid stage program review |
All areas probed thoroughly with the focus on ensuring everything is in place to start delivering the required outcomes. |
|
End stage program review |
This section would not normally apply at program closure stage but some of these areas may need to be considered. |
|
Areas to probe |
Evidence expected |
|
6.1 |
Are the funds available to undertake the next phase? |
|
6.2 |
Are the program’s resources, including inter/intra departmental resources, suitably skilled, available and committed to carrying out the work? |
|
6.3 |
Are the plans for the next phase realistic and achievable? |
|
6.4 |
Are appropriate governance controls and approvals in place? Has the entity assessed its readiness to proceed to the next stage? |
|
6.5 |
Is the governance framework fit for purpose for the next stage; and is there commitment to support key roles and responsibilities for this program within current corporate priorities? |
|
6.6 |
Are the required skills and capabilities for this program available, taking account of the entity’s current corporate commitments? |
|
Implementation Readiness Assessments
The IRA is designed to assess the program/project implementation strategy against its specified objectives, provide early identification of any areas that may require corrective action, and increase confidence that a program/project implementation strategy is effective and conclusive.
Key focus areas for IRA reviews
The IRA focuses on six key focus areas to determine how well an entity is planning for future implementation of a policy initiative.
The key focus areas have been designed to be applied to a range of proposals and not every element in the table may be applicable to a particular proposal. The Assurance Review Team will focus on the areas specific to a policy initiative and flag areas which are not relevant as ‘not applicable’. An assessment between ‘Low, Medium or High’ will be made against each of the key focus areas reflecting the level of issues and risks affecting successful implementation.
The summation of all the issues identified in each key focus area will be included in the body of the report and will establish an overall assessment on whether the initiative is likely to be implemented successfully.
1. |
Policy design |
1.1 |
Has there been a systematic focus on implementation during the policy development stage? |
1.2 |
Is there a clear and concise description of the program/project objectives and how it will work? |
1.3 |
Is this commonly understood between relevant entities? |
1.4 |
Have all delivery challenges been identified and documented? |
1.5 |
Have the critical implementation assumptions been identified? |
1.6 |
Are the assumptions supported by data? |
1.7 |
Have the assumptions been tested? |
1.8 |
Is there a process for monitoring and retesting the assumptions during the design and implementation phases? |
1.9 |
Has a sensitivity analysis been applied to the assumptions and how has this been reflected in program/project design? |
1.10 |
Are arrangements in place to adequately resource implementation planning for the program/project? |
1.11 |
To what extent does the entity consider this is the best way to achieve the policy objectives? |
1.12 |
Has consideration been given to alternative/other options for program/project delivery? |
1.13 |
Have all the options under consideration been adequately tested? |
1.14 |
Is this a demand driven program/project? |
1.15 |
What are the key factors that may influence demand? |
1.16 |
How will funding be impacted by changes in demand? |
1.17 |
How will this emerging cost be managed? |
1.18 |
Is this policy initiative outside the entity’s traditional area of expertise or experience? |
1.18.1 |
If so, how will additional expertise and experience be sourced? |
1.19 |
Has the financial impact of the policy been formally costed? |
1.19.1 |
If so, has the costing been agreed with Finance? |
1.20 |
Is the financial profile appropriate? |
1.21 |
Is there a need for additional capital as well as recurrent costs? |
1.22 |
How have these capital costs been assessed? |
1.23 |
Does the entity have a property management plan in place? |
1.24 |
Are there any gaps in critical information? |
2. |
Implementation planning |
2.1 |
Is there sufficient time to safely implement the program/project? |
2.2 |
Is there an implementation plan? |
2.3 |
Is there a detailed program/project timeline including critical paths, dependencies and milestones? |
2.4 |
Is implementation of the program/project appropriately phased over the forward years and does it reflect manageable stages? |
2.5 |
How have lessons from similar programs/projects been drawn upon? |
2.6 |
Is there a Program/Project Management Office? |
2.7 |
Does the program/project require a corporate Program/Project Management Office function? |
2.8 |
Are there structured change management strategies in place? |
2.9 |
Do financial reporting arrangements provide effective monitoring and forecasting of expenditure? |
2.10 |
Is there an overall ICT plan? |
2.11 |
Has the ICT plan been subject to external scrutiny? |
2.11.1 |
If so, what feedback has been provided? |
2.12 |
How will the appropriate ICT expertise/resources be marshalled? |
2.13 |
Are the ICT requirements clearly within the entities’ existing competencies? |
2.13.1 |
If not, are they novel or leading edge? |
2.14 |
Are there opportunities to leverage existing or developing ICT capability within the Commonwealth? |
2.15 |
Do implementation plans provide for structured testing of IT systems, and formal acceptance sign-off, prior to system implementation? |
2.16 |
Does the entity have the necessary skills available or can they be sourced? |
2.17 |
Does the entity have the capacity to implement the policy without impacting negatively on existing programs/projects? |
2.18 |
Are there appropriately skilled and experienced financial management personnel available? |
2.19 |
Is delivery of the program/project dependent on another government entity? |
2.19.1 |
If so, how mature are the arrangements for accessing this support? |
2.20 |
Does the service delivery entity have demonstrated capability to deliver? |
2.21 |
Has consideration been given to cross-organisational functions and funding arrangements? |
2.22 |
Is delivery of the program/project reliant on third party suppliers? |
2.23 |
Is the external to government supplier market limited and/or very specialised? |
2.24 |
Has a high-level procurement strategy been developed? |
2.25 |
How does the procurement strategy support value for money? |
2.26 |
Does implementation depend on complex or innovative procurement processes? |
3. |
Governance arrangements |
3.1 |
Is there an established and documented formal governance structure in place? |
3.2 |
Is the governance plan appropriate for this policy initiative? |
3.3 |
Do governance arrangements for the program/project require a change in corporate governance arrangements? |
3.4 |
Does the size and complexity of the initiative require a formal or separate governance structure? (E.g. steering committee or task force to assist in implementation). |
3.5 |
Do members of the formal or separate governance structure have the required skills and are they at the right level? |
3.6 |
How will the entity’s senior management officials monitor the development and implementation of the initiative? |
3.7 |
Is there a senior official responsible for the delivery of this policy initiative? |
3.8 |
How has senior management commitment for this policy initiative been demonstrated? |
3.9 |
Is there independent input to the governance process? |
3.10 |
Will the governance arrangements be effective in providing leadership and direction for the delivery of the policy initiative? |
3.11 |
Have roles and responsibilities been defined? |
3.12 |
Are financial delegations well understood and appropriate? |
3.13 |
Are there records of key program/project design decisions including the basis for those decisions? |
3.14 |
Are appropriate monitoring and reviewing processes in place? |
3.15 |
Are monitoring arrangements linked to the critical implementation risks? |
3.16 |
Is the policy initiative likely to involve probity issues? |
3.17 |
If so, has a probity plan been prepared? |
3.18 |
How will key probity concerns be managed? |
4. |
Risk management |
4.1 |
Is there a risk management policy within the entity, including processes for the monitoring, escalation and ownership of risk factors on a day-to-day basis? |
4.2 |
Is there a specific risk management plan for the program/project? |
4.3 |
Is there a common understanding of the risks and who carries the specific risks between the delivery entity, central entities and partner entities? |
4.4 |
Have all the risks been identified? |
4.5 |
Have treatments for these risks been identified? |
4.6 |
Is the risk management plan based on current and appropriate guidelines? |
4.7 |
Are there contingency plans, including an exit strategy, for extreme risk implementation scenarios? |
4.8 |
Is there an effective escalation strategy in place and how will this work in practice? |
4.9 |
Has the accountable authority been briefed on the risks to implementation? |
4.9.1 |
How has this been documented? |
4.10 |
Has the responsible Minister and government been briefed on the risks to implementation? |
4.10.1 |
How has this been documented? |
4.11 |
Are there planned measures in place to deal with fraud and abuse? |
5. |
Stakeholder management and communications |
5.1 |
Have all the key stakeholders been identified? |
5.2 |
Is there clear responsibility for stakeholder management, and the outcomes required from stakeholder engagement? |
5.3 |
Have stakeholders already been engaged? |
5.4 |
What is the strategy for stakeholder engagement? |
5.5 |
Have senior management provided input into the analysis of key stakeholders? |
5.6 |
How has stakeholder feedback been reflected in program/project design? |
5.7 |
If stakeholder resistance is likely, how will it be managed? |
5.8 |
How will stakeholder expectations be managed? |
5.9 |
How do the key stakeholders interact with governance arrangements? |
5.10 |
Is there a communication plan encompassing both internal and external stakeholders? |
5.10.1 |
Is there an alignment between the communication strategy and stakeholder analysis? |
5.11 |
Are resources targeted to maximise the necessary messages to all stakeholders? |
5.12 |
Is whole-of-government consultation required? |
5.13 |
Are arrangements in place to ensure shared implementation activities are aligned? |
6. |
Evaluation and performance |
6.1 |
Is there a clear articulation of the expected outcomes and benefits and how these will be realised? |
6.2 |
Have key performance indicators been developed for the program/project? |
6.3 |
How will the key performance indicators be applied in on-going program/project management? |
6.4 |
Will the policy initiative be formally evaluated? |
6.5 |
Is there an evaluation strategy? |
6.6 |
Do arrangements for data collection support effective evaluation? |
6.7 |
How will results of evaluation be used to inform future implementation? |
Glossary and definitions
Assurance of Action Plan (AAP) - An additional assurance offering that will provide an opportunity for entities to take remedial action and receive early assurance that their action plan is addressing issues.
Assurance Reviewer - An Assurance Reviewer is an individual with extensive relevant experience in a particular field, who has been engaged by ARU. Reviewers may be sourced from the public or private sectors, depending upon their suitability and availability for a project
Assurance Reviews Unit (ARU) - ARU is responsible for supporting the Assurance Reviews function within Finance.
Blended reviews - A combined focus of program and project reviews, which simultaneously provides program strategic alignment and milestone delivery assurance.
Delivery Confidence Assessment (DCA) - The DCA represents the collective view of the Assurance Review Team on the likelihood of overall success for the program/project. This will be their consensus view and the reasons for it will be set out in the review report. The DCA is designed to be considered alongside the specific recommendations the Assurance Review Team has provided. The DCA uses a five-tier rating system (Red, Amber/Red, Amber, Amber/Green, and Green) and the criteria are outlined under ‘The review report’ section.
Enhanced Notification (EN) - A process to ensure that key stakeholders are assured of the earliest possible warning of increased risk of delivery failure, and provide an opportunity to initiate prompt action to get things back on track.
Entity - The non-corporate Commonwealth entity which has responsibility for delivery of the program/project that is the subject of the review.
Gate(s) - Gates are particular point(s) in a project’s lifecycle when a Gateway review is undertaken for projects.
Gateway - The United Kingdom’s Office of Government Commerce Gateway Review Process™, which is a program/project assurance methodology that involves short, intensive reviews at up to six critical stages in a programs/projects lifecycle.
Implementation Readiness Assessment (IRA) - A review that provides assurance to the responsible minister, accountable authority and the cabinet on the implementation readiness of certain proposed government programs/projects
Information Communications Technology (ICT) - is often used as an extended synonym for information technology (IT) but is usually a more general term that stresses the role of unified communications and the integration of telecommunications (telephone and wireless signals), computers, middleware as well as necessary, software, storage and audio visual systems, which enable users to create, access, store, transmit and manipulate information.
Infrastructure - Basic facilities, services, and installations needed for the functioning of a community or society, such as transportation and communications systems and public institutions such as schools. Infrastructure projects typically involve the introduction or enhancement of facilities, services, and installations to meet a particular community need.
Lessons Learned - A tool for sharing information on how program/project obstacles were overcome, and what could be done better on the next phase/gate or next projects. Finance produces a ‘Lessons Learned Report’, on a periodical basis, which promulgates in an aggregated form, observations and evidence of good practice gained from conducting Assurance Reviews.
Onsite review - The Assurance Review conducted over five working days, at the entity’s premises. The onsite review includes an examination of the requested documentation and interviews with key program/project participants.
Outcomes - Identify the key purpose or objectives of the entity, including strategies central to the achievement of the outcomes over the Budget and forward years.
Public Governance, Performance and Accountability Act 2013 (PGPA Act) - entity or entities refers, for the purpose of this Guide, to Commonwealth entities as set out under section 10 of the PGPA Act).
Planning meeting - A meeting between the Assurance Review Team, the entity’s SRO, project manager and other key stakeholders. The planning meeting is held to clarify the intent, scope, logistics for the Assurance Review.
Private financing - A form of government procurement involving the use of private sector capital to wholly or partly fund an asset—that would otherwise have been purchased directly by the government—used to deliver Australian Government outcomes. Private financing is generally an option to be considered only for major asset and infrastructure procurements.
Procurement - The acquisition of property, goods or services through purchase, hire, lease, rental or exchange.
Program - In the context of an Assurance Review, programs can represent a series of interrelated projects or activities with a common objective, or a broad framework or policy concept that may result in a series of largely independent smaller projects (potentially at different stages of implementation). A program is likely to have a life that spans several years. A group of government-mandated activities that contribute to a common strategic or operational objective that can clearly be linked to an outcome statement as articulated in the Appropriation Acts.
Project - Process undertaken by a project team, which is guided by a project management framework, to achieve a new one-off product or service within a finite period of time, in contrast to ongoing work.
Project Management Framework - A set of integrated, cohesive and related tools, procedures and techniques that can be used to guide the execution of a project.
Project Manager - The official within or engaged by the entity, with overall responsibility for the delivery of a project. The project manager typically reports to the SRO.
Project Team - The team of individuals engaged by the entity to assist the project manager in the delivery of a project.
Property - Every type of right, interest or thing that is legally able to be owned. This includes, but is not restricted to, physical goods and real property as well as intangibles such as intellectual property, contract options and goodwill.
Review report - The report issued by the Assurance Review Team at the conclusion of the Assurance Review to the entity’s SRO.
Review Report Template - The template issued by ARU to the Assurance Review Team to assist in the preparation of the Assurance Review report.
Review Team - A team of expert reviewers engaged to undertake an Assurance Review. Members of an Assurance Review Team must not have worked on the program/project under review, except in the capacity as a reviewer at previous gates or stages.
Review Team Leader (RTL) - The member of the Assurance Review Team engaged for their expertise and experience, who has broader responsibility for managing the Assurance Review and is the key point of contact with ARU and the entity’s SRO.
Review Team Member/s - The member/s of the Assurance Review Team engaged for their expertise and experience to contribute to the assessment of a program/project’s progress against its stated objectives.
Reviewer training - Approved training programs that must be completed as part of the process of becoming an Assurance reviewer.
Risk Potential Assessment Tool (RPAT) - The risk assessment tool used by entities to assess the inherent risk of NPPs. It provides a standard set of high level criteria against which SROs and entity project managers can assess the characteristics and degree of complexity of a proposed program/project. It also assists an entity to determine a risk rating for a proposal or package of proposals and communicate the potential risk of that proposal to the responsible minister before seeking the cabinet’s agreement. The risk rating will help determine whether additional assurance such as Gateway reviews or Implementation Readiness Assessments may be required.
Senior Responsible Official/SRO - The official within the entity that has overall accountability for the realisation of the program/project outcomes and objectives for the program/project that is under review.
Stage(s) - A particular point(s) in a program’s lifecycle when an Assurance Review is undertaken.
Stakeholder - An individual or entity who is either potentially affected by the program/project or who has a potential effect on the program/project.
Footnotes
[1] OGG Gateway® is referred to in this document as ‘Gateway’. Gateway® is a Trademark of the UK office of Government Commerce.
[2] www.finance.gov.au/assurance-reviews/risk-potential-assessment-tool
[3] OGC Lessons Learned Effective Project Assurance (June 2009)
[4] https://www.anao.gov.au/sites/g/files/net616/f/2014_ANAO%20-%20BPG%20Po…
[5] https://www.anao.gov.au/sites/g/files/net616/f/ANAO_Report_2010-2011_12…
[6] The Department of the Prime Minister and Cabinet (PM&C), Guide to Implementation Planning August 2001, p. 10