Balanced verdict: AI is excellent at suggesting cleaning approaches. It's dangerous when applying changes unsupervised — because it can silently change values it shouldn't, hallucinate plausible-looking fills for missing data, and behave inconsistently across large files. Use AI as an advisor; use deterministic rules with a review step for the actual changes.
Where AI Genuinely Helps
Inferring Categories from Messy Free Text
Given a column of free-text company descriptions, an LLM can classify them into industry categories with impressive accuracy. "Provides IT support services to SMBs" → "IT Services". "Manufactures automotive components" → "Manufacturing". This would take hours manually and is genuinely error-prone even for a human.
Suggesting Standardization Patterns
Paste a sample of 20 values from a messy column into ChatGPT and ask "what standardization patterns do you see?" The model will often identify the variants correctly: "These appear to be phone numbers in multiple formats — some have country codes, some have leading zeros, some have dashes." This is a great first-pass analysis before you write your cleaning rules.
Explaining Anomalies
"This column has a value of 99999 — what might this mean?" A knowledgeable model can recognize that 99999 is often a placeholder for unknown/null in legacy systems. It can spot that a -1 in a quantity column might indicate a return or deletion code. This kind of domain-pattern knowledge is useful for the investigation phase of data cleaning.
Generating Transformation Code
"Write me a Python/Excel formula to standardize these phone numbers to E.164 format" — models are good at this. The generated code is deterministic and auditable, which means it's the right kind of AI assistance: it helps you write the rule, then the rule runs predictably.
Sohovi profiles your datasets for quality issues in minutes — see what's broken before it breaks your pipeline — try Sohovi free.
Where AI Quietly Destroys Your Data
Silently Changing Values It Shouldn't
This is the most dangerous failure mode. Ask an LLM to clean a dataset and it may:
- "Fix" a product code it thinks looks like a typo ("TSTPRD001" → "TESTPRODUCT001") when TSTPRD001 was a valid internal code
- Normalize a name ("Dr. Chan" → "Dr. Chan MD") when the original had no MD
- "Correct" a price that looks wrong to the model but was intentionally set that way
These changes are coherent — they look plausible. That's what makes them dangerous. You won't notice them in a spot check the way you'd notice obvious corruption.
Example of the real failure:
A real-ish ChatGPT session cleaning a contact list:
You: "Clean these company names — remove formatting inconsistencies" ChatGPT applies: "3M Company" → "3M", "AT&T" → "AT and T", "McKinsey & Company" → "McKinsey and Company"
The AT&T and McKinsey changes look like "cleaning" but are wrong — they're the official legal names. A dedup that follows will now treat "AT&T" and "AT and T" as different entities.
Hallucinating Plausible-Looking Fills for Missing Data
If you ask an AI to fill in missing values, it will — and the fills will look real even when they're invented.
Sohovi profiles every column in your dataset for completeness and flags the exact rows where values are missing — free to try.
Ask an LLM: "Fill in the missing company names for these contact records."
It will produce realistic-looking company names for blank cells. The problem: it's making them up. A contact from "john@gmail.com" might get assigned "Acme Corp" because that looks like a plausible employer — it's not his employer.
This is the worst failure mode because wrong data that looks right is more dangerous than obvious corruption. You'll use these fills in analysis, segmentation, or outreach — and discover the problem much later, if at all.
Inconsistency Across Chunks of Large Files
LLMs process data in chunks. If your file is larger than the model's context window, it processes sections independently. The same value in chunk 1 and chunk 10 may be normalized differently because the model doesn't remember the decision it made earlier.
"Acme Corp" in rows 1–50 might be standardized to "Acme Corporation". "Acme Corp" in rows 500–550 might stay as "Acme Corp". The inconsistency is the opposite of what standardization is supposed to achieve.
The Privacy Problem
Pasting a spreadsheet with customer names, emails, and phone numbers into ChatGPT means sending that PII to OpenAI's servers. This is a GDPR data processing relationship with no DPA. For any business handling EU, Indian, or California customer data, this is a compliance issue. See: Why Pasting Customer Data into Free Online Tools Is a GDPR Risk.
Sohovi automatically detects PII in your datasets — emails, phone numbers, SSNs — all processed client-side so your data never leaves the browser.
The Right Architecture: AI for Suggestions, Rules for Application
The model that actually works:
-
AI for analysis: Paste a sample of your data (anonymized if needed) into an LLM and ask "what cleaning do you suggest?" Use its suggestions as a starting point.
-
Deterministic rules for application: Translate the suggestions into explicit, auditable rules — Excel formulas, Power Query steps, or Sohovi validation rules. The rules are predictable, repeatable, and reviewable.
-
Human review for exceptions: Any value the rule can't confidently handle goes to a review queue — a human decides. This is how you catch the "AT&T" → "AT and T" mistake before it proliferates.
-
Audit trail: Keep a record of what changed, when, and why. "AI cleaned this" is not an audit trail.
Sohovi's Approach: Deterministic, Previewable, Auditable
Sohovi uses explicit rules, not AI inference, for data transformations:
- You define the rules ("email column must match this pattern")
- You preview what will change before applying
- Changes are tracked in a cleaning log
- Exceptions are flagged for review, not auto-filled
No surprises. No plausible-looking hallucinated values. No inconsistency across large files. No PII leaving your browser.
Frequently Asked Questions
Q: Can I use AI to analyze data quality issues without it changing my data? Yes — this is the safe use case. Paste a anonymized sample and ask for analysis ("what patterns do you see?", "what might cause these errors?"). The model observes and suggests; you decide what to do. No data is modified.
Q: Is there any AI tool that safely cleans data automatically? Some tools use AI to generate and refine cleaning rules (rather than directly transforming data). The key test: does the tool show you a preview of changes before applying them, and does it have a human review step for uncertain cases? If yes, the AI is being used safely as a rule-generator. If the tool auto-applies changes without preview, be skeptical.
Q: What about using AI to write Excel formulas that clean my data? This is excellent. A formula is a deterministic rule — "TRIM(LOWER(A2))" does exactly the same thing every time. Using AI to generate the formula and then applying it manually is the right pattern. The formula is auditable; you can see exactly what it does.
Q: If I anonymize the data first, is it safe to paste into an LLM? Safer — properly anonymized data (see How to Anonymize a CSV Before Sharing It) is not personal data under GDPR, so the processor relationship issue doesn't apply. But the problem of hallucinated fills and silent value changes remains regardless of anonymization.
Use rules you can see, preview every change, keep an audit trail — that's how Sohovi cleans data. No surprises, nothing pasted into a chatbot, no plausible-looking changes that turn out to be wrong. Try it free on your own file.