Data Lineage

Data lineage tracks the origin, movement, and transformation of data as it flows through systems - from its source (an API call, a database, an uploaded file) through each transformation step to its final destination (a report, a model, a dashboard). It answers 'where did this data come from and what happened to it?'

Understanding lineage is essential for debugging data quality issues (where did this incorrect value originate?), assessing the impact of upstream changes (which downstream dashboards will break if we rename this API field?), and meeting compliance requirements that mandate auditability of data processing.

Tools like dbt, Apache Atlas, and OpenLineage track and visualize lineage automatically by recording metadata about transformations as they execute. In regulated environments - GDPR compliance for personal data, financial audit trails - documented lineage showing where data flows is a legal requirement.