Threshold-based data quality means defining specific, measurable minimums for each quality dimension of each dataset — below which quality is unacceptable, above which it's fit for purpose — rather than pursuing perfect quality universally.
"Good enough" sounds like an excuse for low standards. In data quality, it's the sophisticated approach. Chasing perfection on every field wastes resources better spent improving the fields that actually matter.
Why "Perfect" Data Quality Is the Wrong Goal
Perfect data quality is unachievable in practice. Some customers genuinely don't have phone numbers. Some addresses are legitimately ambiguous. Some fields are optional by design.
The right goal is: data quality sufficient to support the decisions and processes that depend on it — not data quality that is technically perfect.
How to Define "Good Enough" for Each Field
For each field, answer these questions:
Clean candidate spreadsheets automatically — Sohovi spots gaps, duplicates, and format errors instantly — try Sohovi free.
What is this field used for? A field used in every customer-facing communication has very different quality requirements than one used only in annual analytics.
What breaks if this field is wrong? If the email field is wrong, campaigns bounce and sender reputation suffers. If the "preferred language" field is wrong, the customer might get an email in the wrong language but still receive it.
What is the current quality level? Your threshold should be calibrated to your actual baseline.
Sohovi tracks quality trends across runs and alerts you when a metric — null rate, duplicate count, score — moves outside its normal range.
What is the cost of improving from current level to threshold? The business case for improvement is: cost of quality failure × frequency, vs. cost of the quality improvement program.
Applying Threshold Logic Across the Organization
When thresholds are defined for each critical field:
- Monitoring becomes meaningful: Alerts fire when quality falls below a defined minimum, not based on arbitrary targets
- Improvement prioritization becomes data-driven: The biggest gaps between current quality and threshold are the highest-priority projects
- Quality reporting becomes honest: "12 fields are below threshold, 47 are above threshold"
Frequently Asked Questions
Q: What is threshold-based data quality? Threshold-based data quality defines specific, measurable minimums for each quality dimension of each important field. It replaces the goal of "perfect quality" with the more achievable goal of "quality sufficient for its intended use."
Q: How do I decide what threshold to set for a field? Consider the business impact of failures and the cost of remediation. Use your historical quality baseline to calibrate the threshold so it catches genuine degradation without generating constant false alarms.
Q: Is "good enough" data quality a lower standard? No — it's a more specific standard. "Good enough" defined rigorously is more actionable than "as good as possible" defined vaguely. A threshold of 98% completeness for the customer email field is concrete and defensible.
Q: What happens when data quality consistently exceeds its threshold? Nothing changes operationally — the threshold is being met. If quality consistently exceeds the threshold by a large margin, consider whether the threshold should be raised to capture the new normal.
Q: How do threshold-based approaches handle once-acceptable data that degrades over time? This is exactly what thresholds are designed to catch. As data quality degrades below the threshold, alerts fire. The threshold acts as the floor below which quality becomes unacceptable.
Q: Should the threshold be the same for the same field across different datasets? Not necessarily. An email field in your marketing contact database may need a higher threshold than the same field in a historical archive dataset. Thresholds should be set based on how the data is used.
Q: How does threshold-based quality relate to quality dimensions like completeness and validity? Thresholds are applied per dimension per field. An email field might have a completeness threshold of 98% and a validity threshold of 97%. These are separate thresholds for separate dimensions.
Q: What's the risk of setting thresholds too low? Low thresholds allow genuinely poor quality data to pass without triggering action. Review thresholds periodically against actual business impact.
Q: What's the risk of setting thresholds too high? High thresholds generate frequent alerts for normal variation, creating alert fatigue. Teams stop responding to alerts because they're always present.
Q: How do thresholds relate to SLAs for data quality? Data quality SLAs express thresholds as formal commitments — often between data engineering teams and downstream consumers. A threshold becomes a data quality SLA when formalized as a commitment with defined consequences for violation.
The goal of data quality isn't perfection — it's fitness for purpose. Setting explicit thresholds is what makes that goal concrete, measurable, and actionable.
