New research into machine learning fairness raises an important question for organisations using AI in decision-making: how reliable is the audit itself?
Machine learning models are increasingly being used to support decisions that have real consequences for people, from healthcare and financial services to fraud detection, risk assessment and credit decisions.
As their use grows, so does the need to understand whether those models are making decisions fairly.
Typically, organisations address this through a fairness audit. A model's outcomes are compared across different groups, the gap is measured against an agreed threshold, and the model receives a result: pass or fail.
But there is a question underneath that process that receives far less attention.
Our research suggests that, surprisingly often, you may not.
The problem with a simple pass or fail
Imagine a model is being tested for potential bias.
The organisation has established that a score above 0.80 constitutes a pass. The model is tested and scores 0.81.
It passes.
Now run exactly the same test on another representative sample from the same population. This time the score is 0.79.
It fails.
Nothing about the model has changed. The test has not changed. The threshold has not changed.
Only the sample has changed.
That matters because a threshold draws a hard line through a measurement that naturally moves depending on the people included in the sample. When a result sits close to that line, whether it passes or fails can partly depend on the sample being tested.
Yet the final audit report may simply say "pass".
It gives the person reading it little indication of how close that result was to failing, or how likely the result is to change if the test is repeated.
Why fairness matters in machine learning
Machine learning models learn patterns from historical data.
Take a credit-scoring model. It might be trained on years of applications containing information such as income, existing debt, repayment history and employment history, alongside whether each borrower ultimately defaulted.
From that history, the model learns which characteristics tend to predict an outcome.
The challenge is that the model also inherits characteristics of the data it learns from.
Historical decisions may contain existing bias. Some groups may be underrepresented in the data, making predictions for those groups less reliable. And seemingly neutral variables such as postcode or employment type can sometimes act as proxies for characteristics that were never explicitly provided to the model.
At scale, that becomes important.
An individual decision may not look unusual. Patterns of unfairness can become apparent only when outcomes are examined across a much larger population.
That is one of the reasons fairness auditing has become an important part of responsible AI.
But for an audit to provide meaningful assurance, organisations also need confidence that its result is reliable.
Testing the reliability of the test
This is the problem we set out to investigate through VFR-Audit.
Rather than treating a fairness audit as a single pass-or-fail event, VFR-Audit asks what happens when the assessment is repeated across different statistically valid samples of the same population.
The central measure is what we call the Verdict Flip Rate, or VFR.
Put simply, it measures how often the audit changes its mind.
If repeated testing consistently produces the same result, the verdict is relatively stable.
If it regularly moves between pass and fail, the organisation has a very different kind of result on its hands, even though both scenarios may appear as a simple "pass" in a conventional report.
VFR-Audit also looks at two related questions: whether there is enough data for the assessment to be meaningful, and whether the result remains consistent across different locations or populations.
Together, these provide a better picture of how much confidence should be placed in an audit result.
What we found
We tested the approach using 925,128 hospital records from 441 hospitals, across 12 machine learning models and 336 separate fairness checks.
The results were significant.
Of the 336 checks, 77 were unstable enough to represent a practical problem. And even after applying a standard bias mitigation technique, 11 of 28 checks continued to flip between pass and fail.
That creates an important distinction.
A team could apply an accepted fairness intervention, see the model clear the required threshold and sign it off in good faith, while still holding a result that may not be reproduced the next time the assessment is run.
That does not necessarily mean the model is unfair.
It means there is not enough evidence to be confident that the verdict is stable.
And those are two very different things.
What could this mean for financial services?
The research was conducted using healthcare data, but the underlying issue is not specific to healthcare.
It can arise wherever a measurement is converted into a yes-or-no decision using a defined threshold.
Financial services provides a useful example.
Banks and financial institutions use models and thresholds across areas including credit approval, pricing, fraud detection, collections and risk management.
Consider a regular fairness assessment of a credit model.
If the model records a result just above an organisation's fairness threshold, knowing that it has technically passed is useful.
Knowing whether that result remains a pass across hundreds of equally plausible samples is potentially much more useful.
The same approach could help organisations answer questions such as:
Is our fairness result stable enough to rely upon?
Do we have enough data to make that assessment confidently?
Does the result hold across different branches, regions or portfolios?
Would the same model receive the same fairness verdict if the audit were repeated next month?
Importantly, VFR-Audit does not require the underlying model to be retrained simply to assess this reliability. It sits around an existing fairness check and tests how stable its outcome is across alternative samples.
Reliability is not the same as fairness
There is an important limitation to make clear.
VFR-Audit tests the reliability of a fairness assessment. It does not determine whether the fairness measure an organisation has chosen is itself the correct one.
A consistently repeatable result can still be based on the wrong measure.
Similarly, our research was conducted on hospital data. The finding that 43.5% of verdicts flipped applies to the dataset and experiments used in this research. It should not be interpreted as predicting the level of instability that would exist within a bank, lender or other organisation.
Different industries also operate under different regulatory frameworks, so applying this approach in a regulated environment requires the appropriate legal, risk and governance input.
What the research demonstrates is a broader issue: a fairness verdict and confidence in that verdict are not necessarily the same thing.
A broader question for responsible AI
As organisations move from experimenting with AI to embedding it into operational and decision-making environments, the way we govern those systems needs to mature with them.
A pass/fail threshold is attractive because it provides certainty.
But complex models and real-world populations rarely fit neatly on either side of a line.
For organisations developing their responsible AI practices, that suggests asking a second question alongside the first.
That shift may seem small, but it changes what an AI fairness audit provides: from a point-in-time verdict to evidence about how much confidence an organisation should place in that verdict.
And as AI becomes responsible for more consequential decisions, that distinction will matter.
About the research
VFR-Audit was developed by Joy, Vo and Chua at Swinburne University of Technology and presented at CIKM 2026 in Rome. The research introduces Verdict Flip Rate and a broader framework for assessing the reliability of machine learning fairness audits.
Preprint: https://arxiv.org/abs/2608.30846
Contact us at discover@nexwave.com.au