Inside a Parquet File

The script below writes the 100,000 seat bookings from Row and Column Storage to one Parquet file. It then reads only two things back: the first and last bytes of the file, and the metadata that describes the rest.

pq.write_table(make_table(), path, row_group_size=25_000)

with open(path, "rb") as f:
    head = f.read(4)
    f.seek(-8, os.SEEK_END)
    tail = f.read(8)
print("pyarrow", pa.__version__, "| file", os.path.getsize(path), "bytes")
Copy