PGPA Rule section 16EA(c)
The performance measures meet this requirement when they provide an unbiased basis for the measurement and assessment of the entity's performance.
In order to avoid false conclusions, performance measures must provide an unbiased basis for assessment of the entity’s performance.
When designing an approach to collecting and/or analysing information, entities need to consider how bias might be unintentionally introduced.
Common sources of bias
Occurs when relevant information is not collected (for example, where information is not collected on activities that make key contributions to fulfilling a purpose).
Occurs when individuals or groups are disproportionately represented in the sample from which information is collected (for example, when key stakeholders are omitted from the sample group).
Occurs when the sample group is aware that it is being observed/tested and changes its behaviour either consciously or subconsciously (for example, when people perform differently because they know they are being observed/tested).
Occurs when the people collecting information have preconceived ideas about how a system should behave or about what the results will be.
Occurs when the process for collecting information is not followed or when errors are made in the recording and analysis of data.
Case Studies and Surveys
Where performance measures are based on case studies or survey results, entities should ensure that measures are supported by clear methodology that explains the basis for selecting case studies and identifies how surveys will be conducted.
In particular, case studies selected ‘after the fact’ (that is, after a particular activity has begun or has been completed, or after the entity’s period for performance has ended) introduce the potential for bias.
Case studies should not be relied upon as a stand-alone measurement unless the scope of the case study is predetermined, the activities clearly stated and the measurement methods determined in advance. This avoids the risk of introducing bias where only favourable case studies that tell ‘success stories’ are selected.
Case studies and details of information to be collected should therefore be decided at the time the corporate plan is developed and before information collection occurs. Sufficient information should be included in the corporate plan to provide confidence to the reader that the selection of case studies and reviews are unbiased.
When considering the use of a contracted research and survey provider the selection process should take account of the qualifications and experience of the provider and the standards under which the provider will conduct work.
This includes ensuring the work is conducted in accordance with:
- international quality standards (for example ISO 20252 Market, opinion and social research, including insights and data analytics — Vocabulary and service requirements)
- information security standards (for example ISO 27001 Information security management)
- Australian Privacy Principles (Commonwealth Privacy Act 1988).