Data Retention

A data retention policy defines how long an organization keeps different types of data before deleting or anonymizing it. Retention periods are typically determined by legal requirements (tax records must be kept for 7 years in many jurisdictions), business need (operational data kept while relevant), and privacy obligations (personal data kept only as long as necessary under GDPR).

Retention policies are both a compliance requirement and a data management practice. Deleting data you no longer need reduces storage costs, minimizes breach risk (you cannot lose data you do not have), and simplifies GDPR compliance by avoiding the accumulation of personal data beyond its useful life.

When building pipelines that collect data from APIs - especially any data containing personal information - define retention periods as part of the initial design. Automated deletion jobs, archiving older data to cold storage, and replacing exact values with aggregates for long-term analytics are all valid retention strategies.