Data profiling sounds technical. In practice, it's a structured way of answering a simple question: "What's actually in this dataset?" Here's how the process works, step by step.
Step 1: Examine the Dataset Structure
Before looking at values, understand the shape of the data: How many rows? How many columns? What are the column names? Do the names match what you expect? This structural overview catches the most obvious problems immediately — a customer list with 3 rows, a dataset with column names that don't match the documented schema, or a file that appears empty.
Step 2: Assess Completeness Column by Column
For each column, calculate the percentage of rows that have a non-null, non-empty value. A column that's 97% complete is reliable. A column that's 43% complete should not be used for filtering or analysis without understanding why so much data is missing.
Flag any column with completeness below a threshold appropriate for your use case (often 80–90% for non-critical fields, 100% for key fields like primary email or customer ID).
Sohovi profiles every column in your dataset for completeness and flags the exact rows where values are missing — free to try.
Step 3: Analyze Value Distributions
For each column, look at the distribution of values. What are the most common values? What's the range (min/max) for numeric columns? How many distinct values exist?
A categorical column (like "country" or "industry") should have a small number of distinct values. If "industry" has 400 variants when you expected 20, you have a standardization problem. If "order quantity" ranges from 1 to 50,000 when typical orders are 1–100, you have outliers worth investigating.
Step 4: Run Validity Checks
For columns with defined formats, check what percentage of values match the expected pattern. Emails should match email format. Phone numbers should have 10 digits. Dates should follow a consistent format. Zip codes should be 5 or 9 digits.
Validity checks require knowing what the "correct" format is for each field — either from business rules or from what the majority of values look like.
Step 5: Check for Duplicates
Identify columns where uniqueness is expected (like email address in a customer list) and calculate what percentage of values are unique. High duplicate rates indicate a deduplication problem that needs to be addressed before the data is used.
Sohovi automatically finds every duplicate in your dataset — including near-matches — and shows you exactly which rows are affected.
Step 6: Flag Potential PII
If the data contains personal information (names, emails, phone numbers, addresses), note which columns contain it. This is important for compliance, access control, and data handling policies.
Sohovi automates all six steps when you upload a CSV — producing a complete profile of every column in seconds, including completeness rates, value distributions, format validity, uniqueness scores, and PII detection.
The output of a profile is a clear picture of your data's health — and a prioritized list of what to fix first.
