Data Completeness

What Is Data Completeness?

Data completeness is the property of an archive containing every record it’s supposed to contain from the source system – no tables skipped, no date ranges excluded, no records silently dropped during extraction. It’s distinct from data accuracy, which is about whether the records present are correct; completeness is specifically about whether anything is missing.

Completeness is typically verified through record count reconciliation – comparing the number of records in each source table against the number that landed in the archive – combined with targeted sampling to confirm that specific known records, particularly ones flagged as high-risk or high-value, made it across intact. For structured data with relationships across multiple tables, completeness also has to account for orphaned records: a line item that made it into the archive but whose parent invoice didn’t, for example, which technically satisfies a raw record count but fails a meaningful completeness check.

The stakes on completeness are highest at the exact moment an organization can no longer fix a gap: after the source system has been shut down. A completeness check run before decommissioning is the last opportunity to catch a missing record while the source of truth still exists to compare against. This is why decommissioning plans treat completeness validation as a mandatory gate before shutdown, not an optional quality check.

Key Points

• Different from accuracy. Completeness asks whether everything is present; accuracy asks whether what’s present is correct. An archive can be perfectly accurate and still incomplete if entire records or tables were missed.
• Verified through record count reconciliation. Comparing source and archive record counts table by table is the standard first check, though it must be paired with relationship checks to catch orphaned records that inflate the count without being meaningfully complete.
• Must account for relationships, not just row counts. A complete archive preserves parent-child relationships intact – every line item correctly attached to its invoice – rather than just matching a total row count that could mask broken links.
• The last checkpoint before shutdown is final. Completeness validation has to happen before the source system is decommissioned, because there is no way to retroactively confirm or fix a gap once the original system no longer exists.
• A named gate in any credible decommissioning plan. Decommissioning runbooks that skip an explicit completeness sign-off step are the ones most likely to surface a missing records months or years later, usually during an audit.

Common Use Cases

✅ Pre-shutdown validation sign-off

Before a legacy application is decommissioned, the project team runs a completeness check comparing source and archive record counts across every table, requiring formal sign-off before the source system is approved for shutdown.

🏛️ Audit defense of archived records

When an auditor questions whether an archive contains the full population of transactions for a given period, the organization produces its original completeness reconciliation report as evidence the migration captured everything, rather than re-verifying against a source system that no longer exists.

🔍 Post-migration spot audits

Compliance teams periodically sample specific high-value or high-risk records – large transactions, records under legal hold – against what was documented in the original completeness check, to catch any gap that a pure record-count reconciliation might have missed.

Data Completeness vs. Related Concepts

Completeness is one of several related data quality checks performed during archiving.

DATA VALIDATION

The broader quality assurance process that completeness is one part of. Validation also covers accuracy, formatting, and relationship integrity – completeness specifically answers whether anything is missing.

RECONCILIATION

The specific technique – comparing counts and values between source and target – used to test for completeness. Reconciliation is the method; completeness is the property being tested for.

REFERENTIAL INTEGRITY

Whether relationships between tables are correctly preserved. A record can be present (satisfying completeness) but have a broken relationship to its parent record (failing referential integrity) – the two checks catch different failure modes.