You're merging your company's CRM with an acquired company's CRM. Or migrating from one data platform to another. Or consolidating three team databases into one. You've been told it'll be a few weeks of work. It's going to be much more than that — and most of the extra time will be spent on data quality problems you didn't anticipate.
Database merges are data quality nightmares by default. Here's why — and what you can do about it.
Why Database Merges Create Data Quality Problems
Schema Mismatches
Every database has its own schema — its own field names, data types, and table structure. Merging two databases requires mapping one schema to the other. This mapping is never perfect.
Fields that exist in one database don't have equivalents in the other. Fields that appear to be equivalent have different formats, different allowed values, or different business definitions. A "Customer Status" field in one system with values "Active/Inactive" needs to map to a "Account Type" field in another with values "Open/Closed/Prospect."
Every mismatch is either a transformation with edge cases, a field that must be dropped, or a new field that must be created.
Sohovi profiles your datasets for quality issues in minutes — see what's broken before it breaks your pipeline — try Sohovi free.
Duplicate Entities Across Both Databases
When two organizations merge, many of the same real-world entities appear in both databases — the same customers, the same vendors, the same products. Without a systematic deduplication process, every shared entity becomes a duplicate in the merged system.
A company with 50,000 customers that acquires a company with 30,000 customers doesn't end up with 80,000 unique customers. They might end up with 75,000 — with 5,000 duplicate pairs that represent customers who did business with both companies.
Finding and merging those duplicates is one of the most expensive parts of any database merge.
Sohovi automatically finds every duplicate in your dataset — including near-matches — and shows you exactly which rows are affected.
Conflicting Values for the Same Entity
When the same entity exists in both databases, the two records often have different values for the same fields. One database has a customer's current phone number; the other has their previous one. One has the correct email; the other has a stale one. One has the relationship history from one company's sales team; the other has relationship history from the other company's sales team.
Which record is right? Which history is authoritative? These questions require judgment — not automation — and multiply across every duplicate entity in the merge.
Different Data Quality Standards
The two databases being merged were almost certainly maintained at different quality standards. One company required email for all contacts; the other didn't. One enforced consistent date formats; the other didn't. One had regular deduplication processes; the other accumulated duplicates over years.
The merged database inherits the lower quality standard — not the higher one — unless significant remediation effort is applied to the lower-quality data before merge.
Frequently Asked Questions
Q: Why do database merges create data quality problems? The key reasons are: schema mismatches between the two systems, duplicate entities that exist in both databases, conflicting field values for the same entity across records, and different data quality standards that must be reconciled.
Q: What is the most expensive part of a database merge from a data quality perspective? Entity deduplication — finding and merging records that represent the same real-world entity across both databases. This requires matching on imperfect keys (names, emails, addresses), human review for borderline cases, and careful merge decisions for conflicting values. It scales with the overlap between the two databases.
Q: What is a schema mapping document and why is it critical for database merges? A schema mapping document specifies exactly how each field in the source database translates to a field in the destination database — including field name mapping, data type conversion, value transformation, and handling of fields that have no direct equivalent. Without it, the merge team makes inconsistent decisions that produce systematic data quality problems.
Q: Should data quality remediation happen before or after a database merge? Before, where possible. Cleaning and standardizing data in each source database before merging is dramatically more efficient than cleaning the merged database after the fact. The merged database is more complex, has more entities, and has interleaved quality problems from two sources.
Q: How long does a typical database merge take? Much longer than initially estimated. Technology implementations (mapping, ETL, integration) take weeks. Data quality remediation — deduplication, conflict resolution, field standardization — takes months. Post-merge validation and cleanup takes additional months. The total for a significant merge is typically 3-12 months of part-time effort.
Q: What is the best matching key for deduplication in a database merge? Email address is the most reliable key for contact records — it's unique per person and consistently captured. Company domain is useful for account deduplication. When email isn't available, a combination of name and phone or name and address is the next best option.
Q: What is a "golden record" in the context of a database merge? A golden record is the authoritative merged record created from two or more duplicate records that represent the same entity. It combines the best data from each source — the most complete, most current, most accurate fields from each record.
Q: How should conflicting field values be resolved during a database merge? Define rules for each field: most recent value wins, verified value wins over unverified, or specific system takes precedence. For complex conflicts, route to manual review. Document every decision rule so the resolution is consistent and auditable.
Q: What is a data quality audit before a database merge? A pre-merge data quality audit assesses the quality of each source database independently — completeness rates, duplicate rates, format consistency, value distribution. It provides the baseline for estimating remediation effort and identifies which source database has the lower quality standard that needs the most work.
Q: How do you prevent merge-related data quality problems from recurring? After the merge, establish the governance practices that were missing from whichever source had lower quality standards: named data owners, validation rules, deduplication processes, monitoring. Don't just clean the data — fix the processes that created the quality problems in the first place.
Database merges are data quality projects as much as they are technical projects. The technical implementation takes weeks; the data quality remediation takes months. Budget both correctly from the start.
