Self-Reporting, The Achilles Heel of the Quality Indicator Program

In recent years, data collection has become a cornerstone of governance in the aged care sector, particularly with the implementation of the National Aged Care Mandatory Quality Indicator Program (QI Program).

Introduced to enhance transparency and accountability, the Qi program mandates that aged care providers report on specific quality indicators every three months.

While this initiative aims to improve care standards, the reliance on providers to ‘self-report’ for some of the indicators does rely on providers all following what put simply is an ‘honesty-based’ system – which is where issues can arise.

With self-reporting, the provider has ‘full control’ over what data feeds into My Aged Care, versus what potentially gets omitted, (eg for example; selectively choosing not to report a fall in the Qi data, or tampering/omitting other indicator reporting).

The QI Program requires aged care providers to report on critical aspects of care, such as pressure injuries, physical restraint, and unplanned weight loss to name a few. These indicators are supposed to provide the Department with insights into the quality of care delivered and help identify areas needing improvement.

Last month a related masthead wrote about the significant grey area around the issue of inconsistencies when recording a ‘fall’, its impact on Quality Indicator reporting and the potential for skewed data due to the variability of how things are reported in the industry.

Selective self-reporting, are all providers doing the right thing? 

An unnamed provider spoke to this masthead about their concerns regarding the potential risks posed by the self-reporting system and limited oversight by My Aged Care which allows providers to ‘cherry pick’ what goes in and what stays out of their reporting before submitting.

“We are doing the right thing and have checks and balances in place to make sure the data that gets reported on the frontline is what gets reported back to My Aged Care, but I can see how some providers would be tempted to pick and choose what goes in easily enough”

“Basically, some providers choose to have internal systems to give them full control over the publishing of information”.

One of the issues is providers can choose from various reporting methods, including paper-based records, their own data collection software, or third-party software services – the more independent often the less likely for temptation. The diversity in reporting methods also opens the door to potential inconsistencies and manipulations, with nothing being tracked or traced or audited.

Self-Reporting bias

Self-reporting, whether through paper or digital means, places significant control in the hands of the providers.

Those using paper-based methods or simple tools have full discretion over what they submit to the Department. The lack of oversight, checks, and balances, combined with the absence of routine audits, makes it relatively easy for low-scoring areas to be suppressed.

“Providers that use paper or basic systems have full control of what they submit or don’t want to submit to the department,” the anonymous representative continued. “There’s no oversight, no checks and balances, and quite often, no auditing going on internally because nobody has the time to do the data collection twice just to make sure the initial round was correct.”

The representative shared that “Given the time pressure that managers and staff are under, it’s not inconceivable to think that someone in the chain might omit lower scoring results,” they said.

The Role of Third-Party Data Collection Services

While third-party data collection services offer a level of impartiality and they often have an inbuilt audit log, they are not immune to issues. However, these services typically prevent providers from manually altering results, adding a layer of protection against data manipulation.

The anonymous representative noted that their organisation couldn’t manually remove any input with their third-party service, a feature they now see as beneficial.

Are the indicators all they are cracked up to be? 

For self-reporting to be an effective benchmark or measurement of quality all providers need to be reporting like for like. The above examples raise further questions about the robustness of the QI Program and if it’s all it’s cracked up to be OR if it’s just simply ticking the box to meet the recommendations of the Royal Commission.

While third-party data collection services offer some level of assurance, the potential for manipulation remains. Implementing standardised reporting protocols, regular automated audits via software systems, and stricter penalties for data manipulation could help mitigate these risks.

Furthermore, fostering a culture where providers aren’t fearful of reporting the facts due to the risk of losing funding or having the watchdog on their doorstep.