Batch Processing

Batch processing is the execution of a series of data jobs on a collected group of records at scheduled intervals - nightly, hourly, or on demand - rather than processing each record as it arrives in real time. The system accumulates records, then processes them all at once in a single run.

Batch processing is well-suited for workloads where slight latency is acceptable and processing efficiency matters: nightly ETL jobs that refresh a data warehouse, weekly dataset exports, or end-of-day reconciliation runs. It simplifies error handling and retry logic since the entire batch can be re-run if something fails.

Batch and stream processing are complementary. A common architecture uses batch processing for large historical loads and stream processing for real-time incremental updates - ingesting the full property dataset weekly in batch while applying new listing and price changes in near-real-time.