Most data quality problems fall into a predictable set of categories. The specific records vary, the fields affected vary, the business impact varies — but the underlying problem types repeat across every industry and every dataset size. Here are the 10 most common, with practical fixes for each.
Problem 1: Duplicate Records
What it is: The same real-world entity appears as two or more records — two customer records for the same person, two vendor records for the same supplier.
Why it happens: Multiple import sources that don't deduplicate, users creating new records when they can't find existing ones, system integrations that don't use upsert logic.
Sohovi automatically finds every duplicate in your dataset — including near-matches — and shows you exactly which rows are affected.
How to fix it: Run a deduplication check using email address or phone as the matching key. Merge confirmed duplicates, preserving all unique data from each. Add deduplication checks to all future imports and a "does this record already exist?" check at manual entry points.
Problem 2: Missing Values in Required Fields
What it is: Fields that must be populated for a record to be usable are blank — no email on a contact record, no amount on an invoice, no customer ID on an order.
Why it happens: Optional form fields people skip, imports that don't map all fields, new required fields added after records were created.
How to fix it: Calculate the null rate for each required field. Fill from available sources where possible. Make truly required fields mandatory at entry. For records that can't be filled, flag with a "needs review" status.
Problem 3: Invalid Formats
What it is: Values that don't match the expected format — emails without "@," phone numbers with letters, dates formatted inconsistently.
Why it happens: Free-text entry without format validation, imports from external sources with different format conventions, data collected from multiple regional sources.
Sohovi lets you set up validation rules for any column and instantly see which rows fall outside them — no code or SQL required.
How to fix it: Run format validation rules to flag invalid values. Standardize correctable cases (phone number formatting). Remove or flag uncorrectable cases (emails with no "@" symbol). Add format validation to entry points.
Problem 4: Inconsistent Categorical Values
What it is: The same category expressed differently across records — "Active," "active," "ACTIVE," and "1" all representing the same status.
Why it happens: Free-text entry without a controlled vocabulary, data from multiple systems that use different conventions, historical changes to approved values without back-migration.
Sohovi validates your dataset before it enters the warehouse — catching format errors, nulls, and duplicates at the source.
How to fix it: Profile the distinct values in each categorical field. Create a normalization mapping from all variants to a canonical form. Apply the mapping, then enforce the canonical list via dropdown or validation going forward.
Problem 5: Stale and Outdated Records
What it is: Data that was accurate when collected but no longer reflects reality — a contact at a company they left two years ago, a product at a price that changed six months ago.
Why it happens: The real world changes faster than data gets updated. No refresh process exists. No staleness threshold is defined or monitored.
How to fix it: Add a "last verified" timestamp to records. Define maximum acceptable age for each critical field. Build re-verification workflows for high-value datasets. Use automated enrichment services where available.
Problem 6: Referential Integrity Failures
What it is: Records that reference entities that don't exist — an order linked to a deleted customer, a product with a category code that's not in the category table.
Why it happens: Records deleted without updating related records, imports that don't check for valid foreign keys, system migrations that missed related records.
How to fix it: Run foreign key validation queries to find orphaned records. Investigate the source of each orphan and decide whether to delete, archive, or restore the missing parent record. Add referential integrity checks to import and deletion processes.
Problem 7: Mixed Data Types in a Field
What it is: A field contains both text and numeric values — a "revenue" field containing "$1,234.56" and "1234.56" and "N/A" in different records.
Why it happens: Multiple people entering data in different formats, imports from different source systems, formula outputs that return different types in different conditions.
How to fix it: Strip formatting characters from numeric fields (remove "$", ","). Replace text placeholders ("N/A", "TBD") with null values. Cast the field to the correct type after cleaning. Enforce type at entry going forward.
Problem 8: Cross-Field Inconsistencies
What it is: Related fields that disagree with each other — a "status" of "Cancelled" with a null "cancellation_date," or a "payment_type" of "Credit Card" with no card details.
Why it happens: One field updated without updating its dependent field, different teams managing related fields independently, system limitations that don't enforce cross-field rules.
How to fix it: Define and document cross-field dependencies. Write queries to identify violations. Fix each violation by either correcting the inconsistent field or updating the dependent. Add cross-field validation rules going forward.
Problem 9: Encoding and Special Character Errors
What it is: Garbled text where special characters should appear — "é" where "é" should be, "’" where an apostrophe should be.
Why it happens: A file written in one character encoding (Windows-1252) read as a different encoding (UTF-8), or vice versa.
How to fix it: Detect the correct encoding using a tool like chardet. Convert the file to UTF-8. Validate that previously garbled characters now display correctly. Always specify UTF-8 when saving CSV files going forward.
Problem 10: Schema and Structure Mismatches
What it is: A file has the wrong number of columns, wrong column names, or wrong column order — causing imports to fail or map fields incorrectly.
Why it happens: A source system changed its export format, a file from a new vendor uses a different structure, a manual export from a different view or report.
How to fix it: Run a schema check on every imported file before loading. Compare the actual schema to the expected schema. Return mismatched files to the source for correction. Never try to import a file with structural problems — fix the structure first.
Frequently Asked Questions
Q: What are the most common data quality problems? The 10 most common are: duplicate records, missing values in required fields, invalid formats, inconsistent categorical values, stale/outdated records, referential integrity failures, mixed data types, cross-field inconsistencies, encoding errors, and schema mismatches. These account for the majority of data quality problems in most organizations.
Q: Which data quality problem causes the most business damage? Duplicate records typically cause the most widespread damage because they affect every process that touches the data — marketing sends twice, sales teams double-contact prospects, analytics inflate counts. Stale data causes the most invisible damage because it looks fine but produces wrong conclusions.
Q: Is it better to fix data quality problems reactively or proactively? Proactively, always. Reactive fixes address problems after they've already caused damage — a bounced campaign, a wrong report, a customer complaint. Proactive prevention through validation and monitoring catches problems before they propagate downstream.
Q: How do I decide which data quality problem to fix first? Prioritize by impact and effort. High-impact problems in actively used datasets should be fixed first. Problems that are creating immediate business damage (high email bounce rate, wrong financial numbers) take priority over cosmetic issues.
Q: Can all 10 of these data quality problems be prevented? Most can be significantly reduced through better system design — validation at entry, controlled vocabularies, deduplication checks, foreign key enforcement. Some problems (like stale data) can only be managed, not eliminated, because they arise from the real world changing.
Q: How long does it take to fix a typical data quality problem? Depends on the problem and the dataset size. Simple format standardization on 10,000 records: a few hours. Full deduplication of a 100,000-record CRM: days to weeks. Fixing cross-field inconsistencies in a complex data model: depends on the number of violations and their complexity.
Q: Do all businesses have these data quality problems? Yes. These 10 problems are universal — they appear in every industry, every company size, every type of data. The specific fields affected and the severity vary, but the problem types themselves are consistent.
Q: What is the fastest way to identify which of these problems exist in my data? A data quality profile — a scan of your dataset that shows completeness rates, distinct value distributions, format validity rates, and duplicate counts — surfaces most of these problems in minutes. Sohovi does exactly this for any CSV file.
Q: How do I prioritize fixing multiple data quality problems at once? Use a simple scoring matrix: rate each problem by business impact (1-3) and ease of fix (1-3). Multiply to get a priority score. Fix the highest-scoring items first. This prevents you from spending a week on a cosmetic issue while a critical problem goes unaddressed.
Q: What should I do after fixing data quality problems? Add prevention mechanisms at the source — validation rules, deduplication checks, controlled vocabularies. Schedule periodic monitoring to catch recurrence. Document what was fixed and what changed to prevent it. Measure the before/after to verify improvement.
These 10 problems appear in virtually every business dataset. You probably have several of them right now. Profile your most important dataset and find out which ones — then fix the source, not just the symptom.
If you want to see which of these problems exist in your most important dataset, Sohovi gives you a complete quality breakdown in under a minute — free, no code, no data leaving your browser.
