OBJECTIVE: Check whether a completion claim matches its population and reporting window. Use synthetic records in a worksheet or authorized test environment. Do not create fake requests in a live service or export private participant histories for a public discussion.
1. Freeze the Question
Write the unit being counted, the entry period, the observation date, and the event that qualifies as completion. For example: of distinct eligible requests received this week, how many had reached the defined completed state by Friday?
Keep this definition beside the result. Changing the definition creates a different measure, even when the display keeps the same title.
2. Reconcile a Small Cohort
Create 20 synthetic requests: 12 completed, four open, two withdrawn, and two with unknown outcomes. Assign each request one mutually exclusive state at the observation time. Confirm that the states total 20 and that the defined completion result is 12 of 20, or 60 percent.
Now give one request three submission attempts. The distinct-request count should remain 20. An attempt-based measure may change, but it must be labeled as a different unit.
3. Introduce a Boundary Change
Recalculate after excluding withdrawals. The result becomes 12 of 18, approximately 66.7 percent. No additional request was completed. The changed percentage comes entirely from a changed denominator.
Require the report to disclose that exclusion. Then introduce a completion from the previous month and verify that it cannot silently enter this cohort’s numerator.
4. Check the Headline
Ask someone unfamiliar with the exercise to explain the result. If they interpret unknown outcomes as successful resolutions, revise the wording. Keep the completion figure alongside unresolved counts rather than treating absence of a recorded complaint as evidence of success.
5. Record the Limit
Save the definition, test cases, expected counts, observed counts, and unresolved questions. Passing this exercise verifies the chosen counting rule, not the quality of every completed outcome.
Use Denominator Boundaries for interpretation and The Empty Denominator for the fictional participant perspective.
Place this measurement exercise within the Digital Systems Accountability review when the reported outcome depends on explanation, challenge, or correction.