MapReduce

1. The Incident

After ten years of business, finance asks for "total revenue by product category, all time, by region." The source contains 5 billion order rows, approximately 2 TB, stored in object storage.

The analyst writes a direct script that reads every row and maintains a running total for each category. Apply the arithmetic; one machine reads approximately 100 MB/s, so 2 TB / 100 MB/s = 20,000 seconds. Reading alone requires five and a half hours, before computation begins; the script starts Monday at 9 AM.