Data Observability

Data observability is the ability to understand the health and state of data in a system at any point in time - detecting anomalies, tracking freshness, monitoring schema changes, and alerting on quality degradation before downstream consumers are affected.

The five pillars of data observability are: freshness (is the data up to date?), volume (are the expected records arriving?), schema (did the structure change unexpectedly?), distribution (are field values within normal statistical ranges?), and lineage (which upstream data and transformations affect this output?).

Data observability platforms like Monte Carlo, Acyl, and custom monitoring built on dbt tests give data teams the same visibility into data health that application monitoring gives to software teams. For pipelines that ingest data from external APIs, observability monitors when the API response schema changes, when expected records stop appearing, or when field distributions shift in a way that suggests an upstream issue.