Profile CSV and JSON Data Without Uploading It

2026-03-03

Data profiling reveals shape before analysis: columns or keys, missing values, types, ranges, uniqueness, and suspicious records. For customer exports, incident samples, or unreleased business data, uploading a file to an unknown service can create a privacy and compliance problem. Prefer approved local processing and share only aggregate findings.

The Data Explorer, CSV Viewer, and JSON Viewer perform supported interactions in the browser locally. Browser-local processing reduces transfer risk, but still verify the tool's implementation and your organization's policy; extensions, telemetry, clipboard history, and the device itself remain relevant.

Profile CSV with Python

The standard library can inspect a CSV without sending it anywhere:

import csv
from collections import Counter

path = "sample.csv"
missing = Counter()
distinct = {}
rows = 0

with open(path, newline="", encoding="utf-8-sig") as handle:
    reader = csv.DictReader(handle)
    fields = reader.fieldnames or []
    distinct = {field: set() for field in fields}

    for record in reader:
        rows += 1
        for field in fields:
            value = (record.get(field) or "").strip()
            if not value:
                missing[field] += 1
            elif len(distinct[field]) < 10001:
                distinct[field].add(value)

print("rows:", rows)
for field in fields:
    print(field, {
        "missing": missing[field],
        "observed_distinct_cap": len(distinct[field]),
    })

The distinct count is capped to limit memory, so label it honestly rather than treating it as exact. Python's CSV parser correctly handles quoted commas and newlines; splitting lines by comma does not.

For numeric columns, parse explicitly and count failures:

from decimal import Decimal, InvalidOperation

def parse_decimal(value):
    try:
        return Decimal(value)
    except (InvalidOperation, ValueError):
        return None

Do not silently coerce malformed values to zero. Record a validation count and a few sanitized row numbers, not full sensitive records.

Profile JSON safely

For a moderate JSON array:

import json
from collections import Counter

with open("sample.json", encoding="utf-8") as handle:
    records = json.load(handle)

if not isinstance(records, list):
    raise ValueError("Expected a top-level array")

key_counts = Counter()
type_counts = Counter()

for item in records:
    if not isinstance(item, dict):
        type_counts["<record>:" + type(item).__name__] += 1
        continue
    for key, value in item.items():
        key_counts[key] += 1
        type_counts[f"{key}:{type(value).__name__}"] += 1

print("records:", len(records))
print("key presence:", key_counts)
print("types:", type_counts)

json.load holds the entire document in memory. Use a streaming parser for very large arrays or JSON Lines, and enforce file-size and nesting limits when processing untrusted files. Spreadsheet formulas in CSV cells beginning with =, +, -, or @ can become dangerous if later opened or exported to spreadsheet software; neutralize them at that boundary.

Build a useful profile

For each field, collect:

Presence and missing counts
Observed types and parse failures
Minimum, maximum, and representative quantiles for numeric values
Minimum and maximum length for text
Exact or approximate distinct count
Candidate identifiers and duplicate counts
Date formats, time zones, and impossible dates

Avoid printing top values for names, emails, addresses, free text, tokens, or rare categories. Aggregates can still identify people when groups are small. Apply minimum group sizes and suppress rare combinations according to policy.

Profile a copy with read-only source access, and record a checksum if you must prove which approved input was analyzed. The checksum is an identifier, not anonymization, so limit where it is shared. For repeatable runs, version the profiling script and its settings, then emit machine-readable aggregate results plus a short warning list. Keep raw-value examples out of that report by default.

Work with representative test data

When a bug can be reproduced structurally, create a synthetic dataset with the Mock Data Generator instead of distributing the source. Preserve types, null rates, boundary lengths, and malformed cases without preserving real identities:

[
  {"customer_id":"C-001","amount":12.50,"created_at":"2026-07-19T10:00:00Z"},
  {"customer_id":"C-002","amount":null,"created_at":"invalid-date"}
]

Never claim that replacing names alone anonymizes a dataset. Dates, locations, rare attributes, and stable identifiers can enable re-identification.

Common mistakes

Do not profile a production file in an online notebook, paste rows into chat, or commit samples to Git. Watch for temporary exports, shell history, editor backups, browser downloads, and cloud-synced folders. Use least-privilege access, encrypted storage, restrictive permissions, and an approved deletion schedule.

Finally, validate encoding, delimiter, header duplication, JSON number precision, and null conventions before drawing conclusions. A trustworthy profile states its sample, limits, approximations, and privacy controls—not just its counts.