Object Storage

Object storage is a data architecture that manages data as discrete objects rather than files in a directory hierarchy or blocks on a disk. Each object contains the data itself, associated metadata, and a unique identifier. Amazon S3 is the canonical example of object storage.

Object storage is designed for unstructured and semi-structured data at massive scale. It is highly durable (typically eleven nines), globally accessible, and very low-cost for data at rest. Objects are typically written once and read many times - it is not optimized for frequent small updates.

Object storage is the foundation of modern cloud data lakes. Pipeline outputs - processed dataset files, API response archives, exported reports - are commonly stored in object storage buckets and retrieved via HTTP API calls to the storage service, making them accessible to any tool with network access.