You can merge duplicate customer records without losing data by first identifying all unique information on each duplicate record, copying it to a master record, verifying the combined record is complete and accurate, and only then removing the secondary records — preserving every phone number, address, and interaction history along the way.
The word "deduplication" makes it sound like a simple delete operation. It isn't. Every duplicate record potentially has unique information — a phone number that only appeared on one record, an interaction that was logged against the other, a different address that was updated on one but not the other. Deleting a duplicate without merging first destroys that unique data permanently.
Before You Merge: Understand What Each Record Contains
Before merging any pair of duplicate records, compare them field by field:
Sohovi automatically finds every duplicate in your dataset — including near-matches — and shows you exactly which rows are affected.
Fields that should be identical: Customer ID, email address (the matching key). If these differ between duplicates, investigate why before merging.
Fields that might conflict: Phone numbers, addresses, job titles, company names. Two records for the same person might have a current and a previous phone number — both worth keeping.
Fields that are additive: Interaction history, purchase history, support tickets, notes. All of this should be preserved on the merged record, not discarded.
Fields that might be stale: Last updated date, last contacted date. Take the most recent version.
The Merge Process Step by Step
Step 1: Designate a master record. Choose the record that is most complete (most non-null fields) and most recently updated. This is your golden record — the one that survives.
Step 2: Identify unique data on secondary records. For each secondary record (the duplicates being removed), identify any field that:
- Has a value the master record doesn't
- Has a more recent value than the master record
- Has an older value that's worth preserving as history
Step 3: Update the master record. Copy unique data from secondary records to the master. For conflicting data (two different phone numbers), add a second phone field rather than discarding one — the customer may have both.
Step 4: Update all related records. In a relational system, all orders, tickets, interactions, and notes linked to the secondary records need to be re-associated with the master record. This is the most error-prone step — missing it creates orphaned records or lost history.
Step 5: Mark secondary records as merged, then remove. Don't delete immediately. Mark secondary records with a "merged_into" reference pointing to the master record ID. Keep them in a soft-delete or archive state for a review period, then permanently delete once you've confirmed the merge is correct.
Sohovi shows you the completeness of each field in your contact records — so you can identify which records have the most complete data before designating your master record.
Sohovi profiles every column in your dataset for completeness and flags the exact rows where values are missing — free to try.
What Not to Do
Don't merge without reviewing. Automated merging based on key field matching is useful for high-confidence exact duplicates. But even then, a spot-check of the merged results catches unexpected data issues.
Don't delete before verifying related records. The most common deduplication data loss happens when related records (orders, tickets, interactions) are accidentally left pointing to a deleted secondary record, creating orphans.
Don't merge records that aren't actually duplicates. Two people with the same name at the same company are not duplicates. Always verify on a secondary identifier (email or phone) before merging.
Frequently Asked Questions
Q: What is a golden record in deduplication? A golden record (or master record) is the single authoritative record for an entity, created by merging the best data from all duplicate records. It contains the most complete, most recent, and most accurate version of the entity's data.
Q: What data can be lost during deduplication if not done carefully? Interaction history (calls, emails, notes logged against a secondary record), purchase or order history, support tickets, alternative contact information (a phone number that only appeared on the secondary record), and any field that had a different (possibly more current) value on the duplicate than on the master.
Q: How do I decide which duplicate record to keep as the master? Prioritize: the record with the most non-null fields (most complete), the record most recently updated, and the record created by the most reliable source. If a CRM-sourced record and a manual-entry record are duplicates, the CRM-sourced record is usually more reliable.
Q: What happens to related records (orders, tickets) when I delete a duplicate? In a relational database, all foreign key relationships pointing to the deleted record become orphaned. Before deleting, update all related records to reference the master record instead. This is a critical step that's often missed in simple deduplication processes.
Q: How do I handle a merge where both records have different phone numbers? Keep both. Most CRM and contact database systems support multiple phone numbers per contact. Add the secondary record's phone as an additional phone field rather than discarding it — the customer may actually use both numbers.
Q: Can I automate the merge process? For high-confidence exact duplicates (same email address, same customer ID), automated merging is reasonable with appropriate safeguards (soft-delete first, review period). For fuzzy matches or cases with significant data conflicts, human review before merging is safer.
Q: What is a soft delete vs. a hard delete in deduplication? A soft delete marks a record as inactive/merged without physically removing it from the database. The record is invisible to normal queries but can be restored if needed. A hard delete permanently removes the record. Always soft delete first during deduplication — it's reversible. Hard delete only after verification.
Q: How do I handle CRM deduplication that spans multiple integrated systems? Merge in the system of record first (your CRM), then propagate the merge to integrated systems. This means: update the CRM record, delete the duplicate CRM record, then update all system integrations that reference the deleted CRM ID to use the master CRM ID. This requires coordinating across systems.
Q: What should my deduplication log capture? Date of merge, duplicate record ID, master record ID, who performed the merge, what data was found on the duplicate that differed from the master, and how conflicting data was resolved. This creates an audit trail for future review.
Q: How long should I keep merged (soft-deleted) records before permanently deleting them? 30–90 days is typical. This gives time to verify the merge was correct and recover from errors. After the review period, permanently delete to reduce database size and avoid confusion.
Deduplication done right doesn't lose data — it concentrates it into a single, complete, authoritative record. The merge is the most important step. Do it carefully and your data will be cleaner and more complete than before.
