This guide is relevant to officials of Commonwealth entities assisting with performance reporting.
This guide is available on the Department of Finance website at www.finance.gov.au.
Other relevant publications include:
- Public Governance, Performance and Accountability Act 2013
- Public Governance, Performance and Accountability Rule 2014
- Resource Management Guide No. 126: Government Business Enterprises GBEs
- Resource Management Guide No. 131: Developing good performance information
- Resource Management Guide No. 132: Corporate plans for Commonwealth entities
- Resource Management Guide No. 134: Annual performance statements for Commonwealth entities
- Guide to preparing the Portfolio Budget Statements
- Section 16EA of the PGPA Rule sets out the requirements that all performance measures must meet for the purposes of section 16E of the PGPA Rule in preparing a corporate plan for a Commonwealth entity.
- Further elaboration on a number of aspects discussed in this guide is provided in Resource Management Guide (RMG) 131 – Developing good performance information.
2. Summary of the requirements
3. The performance measures for an entity meet the requirements of section 16EA if, in the context of the entity’s purposes or key activities, they:
Requirements for performance measures
relate directly to one or more of the entity’s purposes or key activities
The performance measures should be directly aligned to what the entity is trying to achieve, as outlined in an entity’s purposes or key activities.
use sources of information and methodologies that are reliable and verifiable
Performance measures should be supported by clearly identified data sources and methodology. Data sources should provide data that is reliable and able to be verified. The methodologies used to assess performance should be able to produce data that is reliable, be applied consistently, and be able to be substantiated.
provide an unbiased basis for the measurement and assessment of the entity’s performance
The performance measures should provide an objective basis for assessment. This means that performance measures, together with the details of data sources and methodologies, should provide confidence to a reader that the basis of measurement is free from bias.
where reasonably practicable, comprise a mix of qualitative and quantitative performance measures
Where reasonably practicable, the assessment of performance should be supported by a mix of qualitative and quantitative performance measures. This recognises that there may be circumstances where an entity will have only qualitative or only quantitative measures.
include measures of the entity’s outputs, efficiency and effectiveness if those things are appropriate measures of the entity’s performance
An entity’s performance measures should facilitate an assessment of how an entity’s activities support the achievement of its purposes, through the measurement of outputs, efficiency and effectiveness.
provide a basis for an assessment of the entity’s performance over time
Performance measures should enable an entity to demonstrate its performance in achieving its purposes or key activities over time. Generally, trends in performance measured on a consistent basis over time will be more informative than standalone or discontinuous measurement of performance
3. Discussion of the requirements
Directly relates to purposes or key activities (16EA(a))
- Each performance measure must relate directly to one or more of the entity’s purposes or key activities. That is, each performance measure should be aligned to what the entity is trying to achiev.
- For example, for an entity with purposes relating to policing and national security, and activities relating to investigating serious crime, performance measures such as:
- Percentage of cases before the court that result in conviction’ and
- ‘Positive return on investment for investigation of crime’
- are ones that relate to the entity’s purposes or key activities.
- Similarly, for an entity with purposes relating to regulation, and key activities relating to monitoring compliance and taking enforcement actions:
- ‘Level of compliance with [specific statutory obligations] by regulated entities’ and
- ‘Proportion of decisions upheld upon review’
- are directly aligned to its purposes or key activities.
- By contrast, for an entity with purposes or key activities relating to the provision of policy advice to Government on legal matters, readers may find it difficult to understand how measures such as ‘Number of visitors to the entity’ or ‘Investment in new technologies ($)’ relate to the entity’s purposes. There is no clear connection between these measures and the purposes or key activities of the entity. Further, it may be difficult for the entity to demonstrate its level of achievement against its purposes through such measures.
- An entity’s corporate plan should provide a clear and understandable link between purposes, the key activities that assist in achieving these purposes and the performance measures that will be used to measure and assess performance.
- The corporate plan should also will provide a reader with information on who will benefit from the activities to be undertaken and how they will benefit.
- The nature of many Commonwealth activities means that other jurisdictions and/or other parties such as the not-for-profit sector often contribute to the achievement of objectives or outcomes pursued by Commonwealth entities. These objectives or outcomes can be viewed as ‘common objectives or outcomes’. The performance measures developed by an entity should be designed to measure the contribution it makes, recognising that the performance of other jurisdictions or parties will impact on the achievement of these common objectives.
- In developing performance measures for activities that contribute to common objectives or outcomes, it is expected that entities will focus on those activities that make a significant contribution to these common objectives or outcomes, rather than contributions that are minor or incidental to their achievement.
- The following example includes information on the contribution that the Department of Infrastructure, Transport, Cities and Regional Development makes to the performance measure on the number of road fatalities.
Example 1 – Demonstrating how performance measures relate to purposes of the entity
The Department of Infrastructure, Transport, Cities and Regional Development 2019-20 Corporate Plan presents the department’s performance measures under the corresponding departmental programs and outcomes, identifying how these are related to the department’s purposes. The performance measures in the extract below from page 16 are relevant to “Program 2.2 – Road Safety”, which is aligned to the department’s purpose of “making travel safer.” These measures are part of a number of measures the department describes as ‘whole-of-community’ measures which focus on Australian community outcomes the department seeks to influence, noting that a number of factors are at play in the delivery of these results. The presentation of these ‘whole-of-community’ measures assists the reader to understand how the performance measures relate to the entity’s purposes and the way the department contributes to each result.
Reliable and verifiable (16EA(b))
- The performance measures must use sources of information and methodologies thatare reliable and verifiable3.
- Performance measures should be supported by clearly identified data sources andmethodologies. Data sources should provide data that is reliable and able to be verified.Methodologies used should be able to produce accurate data, be applied consistently(both by different users at a given point-in-time and by similar users over differentreporting periods), and be able to be substantiated so that the processes followed ingenerating data for each measure are able to be validated.
- Entities should maintain records that properly document the data sources andmethodologies to be used to measure its performance against each of its performancemeasures.
- Data sources may include research and evaluation centres (such as the Bureau ofInfrastructure, Transport and Regional Economics), internal databases or systems(such as payment or call centre systems), or external sources (such as the AustralianBureau of Statistics or OECD reports).
- Methodologies may include participant and stakeholder surveys, benchmarking,evaluation or data mining.
- Entities should avoid using vague language like ‘qualitative assessment’ but ratheridentify a specific methodology (such as accessing results from internal databases,participant and stakeholder surveys conducted by an independent organisation, orcalculations that rely on inputs from specified internal or external sources).
- Entities should avoid using vague language such as ‘timely’ or ‘ready to implement’when describing timeframes and intended results. Precise parameters should beavailable to a reader applying performance assessment. Without this information, thereis a potential for bias in the reported result and it may be difficult for the measure to bemeasured consistently, or be substantiated.
- For example, a measure such as ‘Government measures are legislated andimplemented in a timely manner’ does not make clear what the measures are, what‘timely’ means, nor what implemented means. Does the entity have a timetable forimplementation? The language used should be clear to enable consistent measurementand enable the reader to understand what is being measured and how.
- By contrast, the information presented in Example 2 below allows the reader tounderstand what is being measured and how, and provides confidence that the datacan be substantiated and verified.
Example 2 – Demonstrating that performance measures are reliable and verifiable
The Department of Infrastructure, Transport, Cities and Regional Development 2019–20 Corporate Plan (p 9) includes a range of performance measures for each program. The table below provides details of performance measures, targets over the life of the corporate plan, the sources of data, and the relevant methodology.
Unbiased basis for measurement & assessment (16EA(c))
- To avoid false conclusions, the performance measures must provide an unbiased basisfor assessment of the entity’s performance.
- When designing an approach to collecting and/or analysing information, entities shouldconsider how bias might be introduced.
- Common sources of bias include:
- Exclusion bias - Occurs when relevant information is not collected (for example,where information is not collected on activities that make key contributions to fulfillinga purpose).
- Sampling bias - Occurs when individuals or groups are disproportionatelyrepresented in the sample from which information is collected (for example, when keystakeholders are omitted from the sample group).
- Interaction bias – Occurs when the sample group is aware that it is beingobserved/tested and changes its behaviour either consciously or subconsciously (forexample, when people perform differently because they know they are beingobserved/tested).
- Perception bias - Occurs when the people collecting information have preconceivedideas about how a system should behave or about what the results will be.
- Operational bias - Occurs when the process for collecting information is not followedor when errors are made in the recording and analysis of data.
- Case studies and reviews selected ‘after the fact’ (that is, after a particular activity hasbegun or has been completed, or after the entity’s period for performance has ended)also introduce the potential for bias. Case studies should not be relied upon as a stand-alone measurement unless the scope of the case study is predetermined, the activitiesclearly stated and the measurement methods determined in advance. This avoids therisk of introducing bias where only favourable case studies that tell ‘success stories’ areselected. Case studies and details of information to be collected should therefore bedecided at the time the corporate plan is developed and before information collectionoccurs. Sufficient information should be included in the corporate plan to provideconfidence to the reader that the selection of case studies and reviews are unbiased.
Qualitative & quantitative performance measures (16EA(d))
- An entity’s performance measures must, where reasonably practicable, comprise a mixof qualitative and quantitative performance measures.
- In most cases, it is expected that an entity will have both quantitative and qualitativeperformance measures to capture the multiple dimensions of an entity’s performance.However, it may not always be ‘reasonably practicable’ for the entity to have qualitativemeasures in the context of the entity’s purposes or key activities. For example, incertain limited circumstances, such as in the early stages of the implementation of aprogram or activity, it may be appropriate for an entity to have solely quantitativeperformance measures. As implementation progresses, it could be expected thatqualitative measures may also be developed.
- For a key activity focused solely on policy development, deciding on the appropriate mixof quantitative and qualitative performance measures that are ‘reasonably practicable’should take into account factors such as the cost of data collection, the value of thedata to the entity and the needs of stakeholders.
- For an entity with purposes relating to policing and national security, quantitativemeasures that measure outputs such as ‘percentage of investigations that result in aprosecution or intelligence referral outcome’ may be appropriate. To provide a morecomplete picture of performance, these quantitative measures could be supplemented with qualitative measures such as ‘level of community confidence in the contribution of the entity to law enforcement and security’.
- In the case of service delivery functions, while the speed of processing claims may bean important quantitative measure of performance, the value of this information wouldbe strengthened by combining it with a qualitative performance measure, such as thequality of decision-making based on the results of a survey of key stakeholders, andmeasures that assess the efficiency of claims processing.
- The following examples include a mix of qualitative and quantitative measures to assistthe reader in gaining a more complete understanding of an entity’s performance.
Example 3 – Mix of quantitative and qualitative performance measures
The Old Parliament House 2019-20 Corporate Plan (p 11) includes a mix of quantitative and qualitative performance measures to measure performance. For Strategic Priority 1, quantitative measures are used to report on the number of visitors (both onsite and to the entity’s website) and the number of people participating in programs, and a qualitative measure is used to measure the quality of service (visitor satisfaction). The corporate plan also notes why static targets are used. This assists the reader in gaining a more complete understanding of the entity’s performance in delivering exhibitions, events, digital experiences and programs.
Example 4 – Mix of quantitative and qualitative performance measures
The Department of Veterans’ Affairs 2019-20 Corporate Plan (p 17) includes a mix of quantitative and qualitative performance measures to measure performance. Quantitative measures are used to report on service delivery (for example, the timeliness and quality of service delivery), and a qualitative measure is used to measure the quality of service (customer satisfaction with their service delivery experience).
|No.||Performance criteria||Target||PBS Page||2019
|18.104.22.168||Timeliness: The percentage of new claims processed within 30 days1||Percentage increase over previous year||p.36||✔||✔||✔||✔|
|22.214.171.124||Timeliness: The percentage of pensioner-initiated reviews (PIRs) processed within 10 days1||Percentage increase over previous year||P.36||✔||✔||✔||✔|
|1.1.2||Quality: Correctness rate||New Claims: >95% PIRs: >95%||p.36||✔||✔||✔||✔|
|1.1.3||Client satisfaction: The percentage of clients satisfied with the level of customer service they received when accessing their entitlements2||80%||p.36||✔||✔||✔||✔|
Measures of outputs, efficiency and effectiveness (16EA(e))
- An entity’s performance measures must include measures of the entity’s outputs, efficiency and effectiveness, if these are appropriate measures of the entity’s performance in the context of its purposes or key activities.
- Historically, output measures have been the predominant means by which many entities have measured their performance. However, an entity’s proper use of public resources can reasonably be expected to include the capacity to measure and assess the effectiveness of an entity’s key activities, and to assess the efficiency of these activities, where this is appropriate in the context of the purposes or key activities of the entity. While output measures should continue to be used, particularly where they are a proxy for the measurement of effectiveness, it would generally be appropriate for entities to have a suite of performance measures that consist of a mix of output, effectiveness and efficiency measures.
- For an entity with key activities involving the development and provision of policy advice to government, the development of output and effectiveness measures may be sufficient as efficiency measures may not be appropriate when the cost and reliability of data collection and the needs of stakeholders are taken into account.
- In some cases it may be appropriate to use a proxy measure (that is, an indirect measure of the activity which is strongly correlated with the activity), for example, when data is not available or cannot be collected at regular intervals, or when the cost of gathering and analysing information is prohibitive or outweighs the potential benefits of collecting and reporting on it.
- Where impact of activities is otherwise difficult to assess, output measures may be an appropriate assessment proxy for whether services have reached their intention, at the right time and cost.
- If proxy measures are used, entities should include in their corporate plan an explanation of why they are being used, demonstrate why the proxy measure is suitable and outline their approach to the development of effectiveness and/or efficiency measures in the longer term.
- A discussion of output, effectiveness and efficiency measures follows.
- Output measures assess the quantity and quality of the goods and services produced by an activity (including their volume or quantity).
- Some common types of output measures are:
- Activity - measures the volume of work undertaken. For example, this may be the number of services provided, the number of service recipients, or the number of goods produced.
- Access - measures how easily the intended recipients can obtain a good or service. There are several dimensions to access, such as timeliness of access (for example, measured in waiting times, processing times, time taken to produce an output), affordability of access (for example, the out-of-pocket cost of medical services), and service availability (for example, 24/7 availability of online services).
- Quality - measures how fit for purpose a good or service is. For example, this may be the extent to which outputs conform to certain standards (such as legislative or service standards). There are several dimensions to quality, such as accuracy (for example, accuracy of payments, decisions, etc.), safety of the good or service, and responsiveness to customer needs (for example, levels of customer satisfaction).
- Efficiency is generally measured as the price of producing a unit of output, and is generally expressed as a ratio of inputs to outputs. A process is efficient where the production cost is minimised for a certain quality of output, or outputs are maximised for a given volume of input. In a public sector context, efficiency is generally about obtaining the most benefit from available resources; that is, minimising inputs used to deliver the policy or other outputs in terms of quality, quantity and timing.
- Examples of efficiency measures may include:
- Cost per benefit payment;
- Cost per inspection; or
- Processing cost per grant.
- Key activities that are transactional in nature (such as the processing of welfare or grant payments, the operation of call centres or revenue collection functions) lend themselves to efficiency measurement, in addition to the measurement of outputs and effectiveness to provide a complete picture of the performance of the entity.
- Measures of effectiveness assess how well an entity has delivered on its purposes; that is, whether the activities of the entity have had the intended impact.
- Measures of effectiveness might include:
- the change in literacy rates as a result of activities focused on raising the national reading standards in primary school children; or
- the change in the number of workplace injuries as a result of work health and safety regulation aimed at reducing workplace injuries.
- Although the examples of effectiveness measures offered above are quantitative in nature, effectiveness will not always be able to be measured in quantitative terms. For many activities undertaken by the public sector, qualitative measures will complement quantitative measures. For example, people receiving education to improve literacy skills might be surveyed to understand the impact that an increase in literacy had on their quality of life.
- The following are a number of examples of output, efficiency and effectiveness measures reported on by entities in their corporate plans. These examples are provided to inform entity thinking in the context of their particular circumstances and are not intended to be prescriptive.
Basis for assessment over time (16EA(f))
- The performance measures of an entity must also provide a basis for an assessment of the entity’s performance over time.
- Performance measures should be developed and reported in a manner that assists a reader to understand an entity’s performance over time. Many of the objectives of government are ones that will only be achieved over the medium to long term. Therefore, the ability to measure the efficiency and effectiveness of an entity’s activities over time provides an entity and relevant stakeholders with a more informed view of an entity’s progress in achieving its purposes, enabling an entity to make more informed decisions about the success or otherwise of existing activities. This enables an entity to demonstrate that it is delivering short, medium and/or long term impacts that are essential milestones in the achievement of entity purposes and in the delivery of government priorities over time.
- To be most meaningful, entity performance should be measured in a consistent manner over a period of time, particularly as many activities undertaken by entities have objectives or outcomes that have longer term horizons. Generally, trends in performance measured on a consistent basis over time will be more informative than standalone or discontinuous measurement of performance.
- However, it is recognised that an entity’s performance measures can be expected to change from time to time as entities progressively enhance their approach to the measurement of their performance. Similarly, an entity’s performance measures can be expected to change as programs move from implementation to ongoing activities. As mentioned earlier, at the initial stages of implementation, quantitative measures that measure outputs may be the most appropriate. Once a program or activity is ongoing, it will generally be appropriate for performance measures to be developed that measure effectiveness or impact and, for activities that are transactional in nature, measures that assess efficiency. Entities should therefore attempt to achieve a balance between maintaining consistency in how performance is measured and periodically improving its approach to performance measurement.
- Where changes are made to performance measures, an entity’s corporate plan should contain sufficient detail about the changes made to enable a reader to understand the rationale for the changes and why the changes improve the way performance is measured for the activities concerned.
- The following example from the Department of Education includes performance measures that assess performance over a range of timeframes. The example from the Department of Infrastructure, Transport, Cities and Regional Development also demonstrates how an entity can develop trend data over time.
Example 5 – Assessment of performance over time
The Department of Education 2019-20 Corporate Plan (pp 19-22) presents performance measures that cover a mix of timeframes. For example, the targets ‘At least 18% of domestic undergraduates are from a low socioeconomic background’ and ‘At least 90% of child care payments to all services are accurate’ address the shorter term, while the targets ‘Halve the gap for Indigenous students in year 12 (or equivalent) attainment rates by 2020’ and ‘Lift the Year 12 (or equivalent) or Certificate III attainment rate to 90% by 2020’ address the medium term. Finally, ‘Australia’s share of the world’s top 10% most highly-cited research publications remains above the OECD average’ addresses the longer term.
Example 6 – Assessment of performance over time
The Department of Infrastructure, Transport, Cities and Regional Development 2019-20 Corporate Plan presents performance measures that are used consistently across several reporting periods. One example of this is the performance measure relating to the number of road fatalities (measure number 3 in Figure 6A), which is adopted and identified in the department’s corporate plan over several reporting periods. This consistency in measurement enables the development of trend data, as seen in the department’s 2018-19 annual performance statement (Figure 6B) where a trend line can be seen, showing a trend toward an overall reduction in the number of fatalities over time.
Figure 6A: 2019-20 Corporate Plan (p-9)
Figure 6B: 2018-19 Annual Performance Statement (p-65)
|11. Number of road fatalities|
|Target||1,053 or fewer fatalities||Trend|
|Previous result||In 2017–18 there were 1,222 fatalities: a 14.3% reduction relative to the 2008–10 baseline, which suggests we are not yet on track to achieve the target To be on track to achieve the National Road Safety Strategy target, the reduction in fatalities relative to the baseline should be closer to 25%<|
|2018–19 result||In 2018–19 there were 1,214 fatalities: a 15% reduction relative to the 2008–10 baseline. This suggests we are not yet on track to achieve the 30% reduction target|
Example 7 – Assessment of performance over time
The Department of Social Services 2019-20 Corporate Plan presents performance measures that are used consistently across several reporting periods. One example of this is the performance measure relating to the percentage of recipients who are not receiving income support within 3/6/12 months after exiting student payments. This consistency enables the development of trend data over time, as seen in the department’s 2018-19 annual performance statement (p 26; below). The annual performance statement shows a comparison table of the results for the current and previous two reporting periods, allowing an assessment of overall performance over time.
Percentage of recipients who are not receiving income support within 3/6/12 months after exiting student payments
This indicator measures the reliance of former recipients of student payments on income support three, six and 12 months after leaving student payments. It is a proxy indicator for sustained self-reliance.
 Reliable and verifiable are intended to have their ordinary meanings. Reliable may be defined as ‘able to be relied on; trustworthy’, and verifiable may be defined as ‘able to be proven as true, as by evidence or testimony; able to be confirmed or substantiated’ (Macquarie Concise Dictionary, 7th ed, 2017).
4. Record keeping
- In line with an accountable authority’s responsibility to properly record and explain theentity’s performance in achieving its purposes,4 it is good practice for entities to maintainrecords that support and explain the approach or rationale for how an entity intends tomeasure and assess its purposes. This could be expected to include:
- the rationale for the entity’s approach to the development of its suite of performancemeasures that demonstrates how the entity’s approach takes into account the context of the entity’s purposes or key activities (for example, why certain types of performance measures are included in the entity’s corporate plan and why it may not be appropriate at a particular point in time, or because of the nature of an entity’s key activities, that an emphasis is given to certain types of measures; see the discussion of sections 16EA(d) to (e) above);
- details of the data sources and methodologies to be used to measure the entity’sperformance against each of its performance measures (see paragraph 14 above);and
- where relevant, the entity’s approach to reviewing and its suite of performanceinformation over time.
- A discussion of an entity’s record keeping responsibilities relating to performance is alsoincluded in RMG 132: Corporate plans for Commonwealth entities.