Data Validation

What Is Data Validation?

Data validation is the process of confirming that data extracted and transformed into an archive is both accurate and complete relative to the original source system, performed before that source system is decommissioned and the comparison becomes impossible. It’s the quality gate that sits between transformation and final sign-off in any archiving or migration project.

Validation typically operates on two levels. Automated reconciliation checks compare record counts, sums, and key field values in bulk between source and target — does the total dollar value of all invoices in the archive match the source, does every table’s record count line up. Manual spot-checking then samples a subset of individual records, particularly ones flagged as high-value, high-risk, or legally significant, and confirms a human reviewing the archived version would recognize it as a faithful match to what the original application displayed.

The reason validation has to happen before shutdown and not be treated as something that can be revisited later if a problem surfaces, is that the comparison baseline disappears the moment the source system goes offline. An organization that discovers a data quality issue in its archive a year after decommissioning has no way to verify what the correct value should have been; the only copy of the truth was the system that’s no longer there to check against.

Key Points

• The mandatory gate before shutdown. Validation has to happen while the source system is still available for comparison, once it’s decommissioned, there’s no baseline left to validate against.
• Combines automated reconciliation and manual spot-checking. Bulk record-count and value comparisons catch systemic issues; targeted sampling of individual high-value or high-risk records catches errors that aggregate checks can miss.
• Covers both completeness and accuracy. Validation confirms not just that every record made it across, but that the values within each record are correct, two related but separate checks.
• Produces a durable sign-off artifact. A documented validation report, showing what was checked and by whom, becomes the evidence an auditor or legal team would expect to see if the archive’s integrity is ever questioned later.
• Owned jointly by IT and the business. Technical reconciliation confirms the data moved correctly; business users are typically the ones who can confirm the archived records actually reflect what they remember the source system showing, which is why validation usually involves both groups.

Common Use Cases

✅ Formal sign-off before decommissioning

Before a legacy system is approved for shutdown, IT and business stakeholders jointly review a validation report combining automated reconciliation and manual spot-checks, providing documented sign-off that the archive is ready to become the sole source of record.

🏛️ Producing validation evidence for an audit

When an auditor questions the integrity of archived data, the organization produces its original validation documentation as evidence of a defensible migration process, rather than attempting to re-verify against a source system that no longer exists.

⚠️ Catching extraction errors while they’re still fixable

During validation, a discrepancy is discovered between the source and archived record counts for a specific table — caught while the source system is still running, allowing the extraction to be corrected before shutdown rather than after.

Data Validation vs. Related Concepts

Validation is the umbrella quality process; two more specific terms describe parts of it.

RECONCILIATION

The specific technique of comparing counts and values between source and target systems. Reconciliation is one method used within the broader validation process, alongside manual spot-checking.

DATA COMPLETENESS

One of the two things validation checks for — whether everything that should be present, is. Validation also separately checks accuracy, which completeness alone doesn’t cover.

DECOMMISSIONING PLAN

The overall project runbook that specifies when and how validation must occur, typically as a named, mandatory gate before the shutdown step is authorized to proceed.