Criterion 5. Analyse and report your digital performance

Criterion 5 – Analyse and report your digital performance

Reporting the performance of digital services across government enables transparency and accountability.

Your responsibilities

To successfully meet this criterion, agencies need to:

establish internal processes to support performance data analysis and reporting
report on progress during IOF states and post-implementation performance data, with key reporting requirements in the following states:
- Strategic planning and prioritisation: report on how you intend to implement a monitoring framework (Criterion 1) to your digital service
- Contestability: report (with evidence) that the Digital Performance Standard has been, or will be, applied to the digital service
- Assurance: report (with evidence) how you have applied the Digital Performance Standard to the digital service
- Operations: report (with evidence) on how the digital service continues to meet customer needs
analyse performance results and act on any improvements to the agency’s digital service.

When to apply

Criterion 5 will apply throughout the design and implementation states of the IOF process and once the digital service is made available to the public in the Live environment.

Examples of analysis and reporting requirements will include but are not limited to:

Strategic planning and prioritisation: outline the planned approach to applying the Digital Performance Standard
Contestability: explain the plan and approach for meeting the Digital Performance Standard and when this will be implemented to support the investment proposal (Digital Capability Assessment Process)
Assurance: demonstrate the progress of the ICT investment towards meeting the Digital Performance Standard and delivery milestones
Operations: demonstrate that the ICT investment is continuing to meet customer needs.

As the monitoring framework matures, agencies should aim to achieve best practice, collecting and monitoring more meaningful user-centric metrics.

Questions for consideration

Does the monitoring framework provide meaningful reporting data about ICT investment outcomes?
Does the reporting data say how well we have delivered the benefits as stated in the ICT investment proposal?
What outcomes are expected from the reported data?
How can the reported data be used to improve outcomes for users?
How can best-practice monitoring be applied to the digital services?

How to apply criterion 5

Collect and report meaningful data: Make sure the performance monitoring frameworks and data analytics tools are fit for purpose and provide meaningful reporting data. While there are numerous metrics, calculations and methods to collect data, your choice should prioritise ‘real-time’ user-centric approaches and align with the criteria in the Digital Performance Standard. The data gathered should reflect the true user experience to gain valuable insights. Agencies are required to report ongoing performance data for digital services delivered via IOF-tracked ICT investments once the service is implemented.
Use data to identify the benefits: Use the data collected to identify service benefits. Benefits can include uncovering service inefficiencies by analysing data on digital service performance, unearthing deeper insights into users’ experience, segmenting user data based on user groups to better understand their needs and working in partnership with users to develop user-based solutions. Further qualitative metrics, complementing the quantitative, can add a rich layer of information on underlying factors influencing the user experience.

Guidance to analyse and report your digital performance

Establish internal processes to support performance data analysis and reporting:
- Set up a dedicated team responsible for collecting, analysing and interpreting your digital service data. The team should be equipped with tools and software to make sure data reporting is accurate and comprehensive.
- Select monitoring tools with built-in analysis and reporting features and that easily integrate with existing software applications. This will make analysis and reporting easier.
- Make sure regular analysis and reporting are part of your monitoring framework. See also: Document how you will implement the monitoring framework and Develop processes for continuous improvement.
Reporting using existing reporting mechanisms will make sure your proposal or project is complying with the Performance Standard. This will occur as part of the DTA’s Digital and ICT Investment Oversight Framework process. Depending on your ICT investment your agency will be requested to report information in the following states:
- Strategic Planning and Prioritisation: within your proposal or business case report on how you intend to implement a monitoring framework to your digital service.
- Contestability: report that the Performance Standard has been, or will be, applied to the digital service. For example, as part of the Digital Capability Assessment Process
- Assurance: report how you have applied the Performance Standard to the digital service and made progress with delivery milestones.
- Operations: using the DTA’s existing data collection mechanisms such as the Approved Programs Collection, report on how your digital service continues to meet customer needs post-implementation.
- Apply the steps outlined in Develop processes for continuous improvement, which emphasises an agile approach to continuous improvement based on performance analysis.

Digital Performance Standard