Vector Database Sharding

The Pattern

When the similarity index outgrows one machine, split it: but similarity search breaks Module 3's best rule. A nearest-neighbor query has no key: the closest vectors could live anywhere: so the default is scatter-gather to every shard, and the craft is finding the cases where a routable key sneaks back in.

First, the arithmetic that forces the split: 500 million embeddings × 1,536 dimensions × 4 bytes ≈ 3 TB of raw vectors: and ANN indexes (HNSW-class) want to live in RAM, with overhead on top