The phrase "modern data stack" has become the standard shorthand for the way contemporary data teams build their infrastructure. If you've looked at any data tool's website in the past few years, you've seen diagrams showing tools connected in a pipeline: ingestion → storage → transformation → visualization. Data quality is often shown as a box on the side, connected by a dotted line.
That positioning is wrong. Data quality isn't a side layer — it's a concern at every stage of the stack. Here's how the modern data stack works and where quality fits at each layer.
What Is the Modern Data Stack?
The modern data stack (MDS) is a collection of cloud-native, modular data tools that together handle ingestion, storage, transformation, and analysis of data. Unlike traditional data infrastructure (monolithic ETL suites, on-premises databases), the MDS is characterized by:
- Cloud-native: All tools run on cloud infrastructure. No servers to manage.
- Modular: Each layer is handled by a best-in-class tool, not a single vendor suite.
- SQL-centric: Most transformation and analysis is done in SQL, not proprietary scripting languages.
- SaaS-based: Most tools are subscription software with APIs and connectors.
A typical modern data stack looks like:
Sohovi profiles your datasets for quality issues in minutes — see what's broken before it breaks your pipeline — try Sohovi free.
Ingestion layer: Tools like Fivetran, Airbyte, or Stitch extract data from source systems (Salesforce, Stripe, Shopify, Google Analytics) and load it into the warehouse.
Storage layer: A cloud data warehouse (Snowflake, BigQuery, Redshift, DuckDB) stores the integrated data.
Transformation layer: dbt transforms raw data into analytics-ready tables, applying business logic and quality tests.
BI / visualization layer: Tools like Looker, Tableau, Metabase, or Mode query the warehouse and present dashboards and reports.
Sohovi validates your dataset before it enters the warehouse — catching format errors, nulls, and duplicates at the source.
Orchestration layer: A scheduler (Airflow, Dagster, Prefect) coordinates when each pipeline step runs.
Where Data Quality Lives in the Stack
Layer 1: Ingestion (Source Quality)
The ingestion layer is the first point of entry, and it's where the most preventable quality problems originate. When Fivetran pulls your Salesforce contacts, they arrive with whatever quality they had in Salesforce — duplicates, null fields, inconsistent formats, and all.
Quality concern at this layer: Are the source systems producing clean, consistent data? Are all expected records arriving? Are field formats matching what downstream transformations expect?
What good looks like: Monitoring for row count drops, schema changes in the source, and null rates in critical fields before the data reaches the warehouse.
Sohovi tracks quality trends across runs and alerts you when a metric — null rate, duplicate count, score — moves outside its normal range.
Layer 2: Storage (Schema Enforcement)
The data warehouse enforces its own schema — if a column is defined as INTEGER and a string arrives, the load fails. This provides some quality protection, but modern ELT approaches (load first, transform in place) often store data in very permissive schemas (everything as text) to avoid load failures.
Quality concern at this layer: Over-permissive schemas that allow bad data through. Lack of primary key enforcement. No deduplication before storage.
Layer 3: Transformation (Quality Testing with dbt)
This is where the most active data quality enforcement happens in a well-run modern data stack. dbt's testing framework allows teams to define not-null, unique, accepted-values, and referential integrity tests that run against every model on every pipeline run.
Quality concern at this layer: Tests that aren't written, tests that fail silently, business logic errors in transformation code that produce plausible-but-wrong output.
What good looks like: Every model has at least not-null and unique tests on key fields. Test failures break the pipeline and trigger alerts before bad data reaches the BI layer.
Layer 4: BI / Visualization (Downstream Detection)
By the time bad data reaches the visualization layer, it's already in your reports. The BI layer can catch some quality problems through anomaly detection and data freshness monitoring, but it's the last and most expensive place to fix them.
Quality concern at this layer: Reports built on unvalidated data, metrics defined differently across different dashboards, no alerts when expected data fails to arrive.
Where Does Sohovi Fit in This Stack?
For teams that have a full modern data stack, Sohovi's in-browser quality checks are most useful at the ingestion and source layer — assessing the quality of data files before they enter the pipeline, validating vendor-supplied data before import, and profiling source datasets before a migration.
For teams that don't yet have a modern data stack, Sohovi operates as a standalone quality layer: upload your CSV, get a quality report, understand your data's completeness, validity, uniqueness, and format issues without any infrastructure.
The Small Team Version of the Modern Data Stack
You don't need Snowflake and dbt and Fivetran to have a modern data practice. A reasonable small-team stack:
- Source systems: Your CRM, your e-commerce platform, your accounting software
- "Warehouse": A well-maintained Google Sheets document or a simple SQLite / PostgreSQL database
- Transformation: SQL queries or Google Sheets formulas, applied consistently
- Quality checks: A data quality tool (like Sohovi) run before any major import or report
- Visualization: Google Data Studio, Metabase, or even a Sheets dashboard
The principles — validate before you load, test your transformations, monitor for freshness and anomalies — are the same whether your warehouse is Snowflake or a Google Sheet.
The Bottom Line
Data quality is not a layer you add to the modern data stack after everything else is built. It's a concern at every stage: source quality at ingestion, schema enforcement at storage, automated testing at transformation, and anomaly detection at the BI layer. The businesses with the most reliable analytics aren't necessarily running the most sophisticated tools — they're the ones applying quality discipline consistently at each step.
