A confidence score on a supplier invoice is a per-invoice measure of how sure an AI system is that its proposed posting is correct. A sound score is built from named factors, typically data quality, pattern match, rule compliance, historical consistency and anomaly checks, and it routes each invoice to one of three outcomes: approve and post inside the limits the firm has signed, flag for a named person to review, or stop the invoice until someone investigates.
A confidence score is only as useful as the decisions it drives and the evidence it leaves behind.
A single figure tells you very little. It does not say what the system was unsure about, so the reviewer starts from scratch, and it invites people to treat the number as a verdict rather than a summary of evidence.
A score you can work with breaks down into factors, each of which can be inspected. When an invoice is flagged, the reviewer should see which factor pulled it down and why, in plain language.
Approve. Every factor is strong and the posting sits inside the signed mandate. The system posts it, writes the decision and its evidence to the audit trail, and moves on. Approved invoices are not invisible. They remain available for sampling, and a sensible firm samples them.
Flag. The system has a proposed posting, but something is not clean enough to act on alone. It sends the invoice to the named reviewer with the proposed coding and the reason, for example "historical consistency low: amount well above the previous six invoices from this supplier". The reviewer accepts, corrects or rejects it. Whatever they decide is recorded alongside the original proposal, so the trail shows both the system's view and the person's.
Stop. Something has failed that no score should outweigh. Changed bank details, a suspected duplicate or an unknown supplier are typical. The invoice is held for investigation. A stop is a hard rule, not a low score. A well-designed system never lets strong marks on four factors buy a pass on a failed anomaly check.
The firm does, in the mandate, not the vendor. Where approve ends and flag begins is a control decision, and control decisions belong to the finance owner. Start conservatively, with more invoices flagged than strictly necessary, and widen the approve band only when the record shows flagged items being routinely accepted unchanged. Every change to the thresholds should be a signed revision, with the old version retained.
An MSP receives its monthly cloud licensing invoice from a distributor. The scan is clean and the format matches previous months, so data quality and pattern match are high. The coding is the usual licensing cost nominal, so rule compliance is fine. But the seat count has risen sharply against last month and there is no matching customer order on file, so historical consistency drops and an anomaly check fires.
The invoice is flagged, with the reason stated, to the person who can confirm whether the extra seats were ordered and rebilled to a customer. That is the conversation that should happen, and the score made sure it did.
A confidence score needs evaluating too. Over time, compare flags with reviewer outcomes. If flags are routinely accepted unchanged, the thresholds may be too tight. If corrections cluster on one factor, look there. Databricks, in its State of AI Agents 2026 report, finds that companies using evaluation tools get nearly six times more AI projects into production. Checking how well a system knows what it does not know is part of that discipline.
Mandate's agent, Invoice Maestro, scores every supplier invoice on these five factors and approves, flags or stops it inside a mandate the finance owner signs, escalating unclear cases to a named person and writing each decision to a tamper-evident, append-only audit trail. It runs as a managed service on Xero and Microsoft Dynamics 365 Business Central. To see how the score behaves on your own supplier base, the 60-Day Proof runs read-only and free on your invoices across two month-ends, with a day-60 report you keep: runmandate.co.uk/60-day-proof.
It should not. Checks such as changed bank details or a suspected duplicate are hard rules, and strong results on other factors should never buy a pass. Ask any vendor to show you this in writing.
The proposed posting, the factor or factors that lowered confidence, and the reason in plain language. Their decision is then recorded next to the original proposal.
Rarely, and only on evidence. Review them after each month-end in the early period, then quarterly, and treat every change as a signed revision.