← BackSee all toolsWhen Two Reports Disagree: One Number, One OwnerWorking demo · sample data · shows a method I used, not a real event
Reporting control · quarantine rule

When Two Reports Disagree: One Number, One Owner

When two sources disagree, what do you say? A quarantine rule, a way to tell real movement from re-categorisation and a register of one source per metric.

I built this control around weekly executive reporting at a venture-backed technology company. The figures here are illustrative.

The situation

The dashboard and the database disagreed.

One executive dashboard said a small number of sites were live. The source database said a much larger number. Both could not be right, and leaders were about to act on the smaller one. This is the control I put around that moment. The numbers below are illustrative.

40Live, per the dashboard
100Live, per the source records
60Unexplained gapFreeze, then reconcile

Rule one

When two sources disagree, how much do you say?

Drag the gap between the two sources. The rule decides whether the number is published, published with a caveat, or quarantined.

Source records
Dashboard
Quarantine

Freeze it. Do not publish it as truth.

  • Show the last reconciled value, labelled as such.
  • Name one owner and a reconcile-by date.
  • Tell leaders which decisions must not use the number yet.

Rule two

Is that change real, or did the structure move?

After a cleanup day or an import, week-over-week deltas mostly reflect how records are categorised, not what happened in the field. Pick a scenario to see how each delta is flagged and what I would tell the executive team.

MetricLast weekThis weekChangeRead as
Servers100180+80Structural ‡
Records, all types200300+100Structural ‡
Billing flag set2060+40Structural ‡
Vendor blockers240+38Structural ‡

This is structure, not activity. A cleanup day reclassified records and created a new record type. Show every delta as suspect, name the cause and hold the headline until Finance confirms.

One definition each

The canonical metric register.

A dashboard can only be trusted if each number has one source, one written definition and one owner. Illustrative register.

MetricCanonical sourceDefinition fixed in writingOwner
Live sitesAnalytics mart, built from the source recordsInstalled, activated, reporting data in the last 7 daysAnalytics lead
Recurring revenueReconciled revenue modelBilling now, not contracted, not shippedFinance lead
Open blockersDelivery trackerOne owner, one next action, one dateDelivery lead
Support responseTicketing systemPriority-tiered clock; alert noise routed separatelySupport lead
Automation healthWorkflow run logSuccess rate of critical flows over 7 daysAutomation lead

The notice

Say how much to trust the number.

Template

The notice at the top of every report

Data integrity notice. Figures marked * come from systems still being cleaned. Treat them as directional until Finance confirms them. Deltas marked ‡ reflect re-categorisation, not new activity. The last time billing data was confirmed current was [date].

Four lines, written once and reused. The point is that nobody has to guess how far to trust a number.

Habits

Four habits that keep a report honest.

  1. Quarantine before you explain. A number you cannot defend does not go to executives.
  2. Mark the cause of every big delta. Cleanup, import, schema change or real movement.
  3. One canonical source per metric. Everything else is a view of it.
  4. Fix at the source. Correct the record, then the report, so the error cannot return.

Also on this page

Related tools folded into this page. Each opens in place; tap Close when done.

Inventory Reconciliation: Three Systems, One CountThe same one-number rule applied to stock records that disagreed across three systems: what was found, why, and the controls that stop it recurring.Open section

More in Data, Reporting & Decision Support

See all tools