Late Data in Batch Pipelines

A daily batch run that finishes at the end of day 10 makes one promise: every reading that had landed in storage by then is counted in the totals it writes. It makes no promise that every reading measured on day 10 had landed by then. Suppose a smart meter loses its mobile connection: it keeps measuring, stores its readings, and uploads them when the connection returns, which can be several days later.

Rows that reach storage after the run that should have counted them are called late data