What to Expect in the Harvey System Design Interview

Harvey's system design round asks you to design systems like the ones behind its legal AI products. Reported question themes include document processing pipelines, search systems, collaboration features, high volume APIs, and permission systems. A permission system decides which user may see which document. The round rewards candidates who connect the design to legal users, where confidentiality and accuracy are strict requirements.

Harvey builds generative AI for law firms. Its products read large sets of legal documents and answer questions about them with citations. The design questions come from exactly that work.

The Question Types Harvey Asks

• Document pipelines. Ingest thousands of files, extract text, split it into chunks, and index it. A chunk is a small piece of text sized for a model to read.
• Search and retrieval. Combine keyword search with vector search. Vector search finds text by meaning, using numeric representations called embeddings.
• Permissions and confidentiality. Law firms separate matters strictly. Access control lists (lists of who may access each item) come up often in this round.
• Collaboration systems. Several lawyers edit and comment on the same document at once.
• High volume APIs. Serve model-backed endpoints with rate limits, retries, and predictable latency.

A Worked Example: Legal Document Search

Here is a high level path through a representative question: design a search system over a law firm's documents.

Step 1: Requirements. Ask about scale (documents, users, queries per second) and about freshness. Then state the legal constraints yourself: results must respect permissions, and answers must cite sources. Naming those constraints early is the strongest signal in this round.

Step 2: Ingestion. Files arrive in many formats. A pipeline extracts text, detects structure, splits it into chunks, and writes to two indexes. One index is keyword based, one is vector based. Make ingestion a queue-driven pipeline so a bad file cannot stop the flow. Store the original file too, because lawyers will ask to see the exact source page.

Step 3: Query path. A query runs against both indexes. Merge the results and rank them with a scoring model. Filter by permissions before ranking, not after. Filtering after ranking risks leaking a document's existence to the wrong user.

Step 4: Answering with citations. Pass the top chunks to a language model and require it to cite the chunks it used. Reject answers whose claims lack a supporting chunk. This is the main defense against hallucinations (false statements presented as fact).

Step 5: Evaluation and scale. Keep a test set of real questions with known answers and score every release against it. Track citation accuracy, not just relevance. For scale, cache frequent queries, shard the indexes, and isolate each firm's data completely. Sharding means splitting one index across many machines.

What Interviewers Grade

They grade constraint thinking first. Permissions and accuracy are not optional features in legal software. Candidates who raise them before being asked do well.

They grade trade-offs spoken aloud. Keyword versus vector search, latency versus quality, cost versus freshness. Name both sides and choose with a reason. A stated assumption you can defend beats a silent guess every time.

They grade communication. Follow a clear order: requirements, high level design, one deep area, risks, measurement. Ask the interviewer which area to expand. This round is one of four onsite rounds, so the loop context in What Is the Harvey Interview Process Like? (Round by Round) is worth reading first.

How to Prepare

• Learn the standard method. Grokking the System Design Interview covers requirements, estimation, and the standard building blocks.
• Study the recurring patterns. Queues, sharding, caching, and search appear in most answers. Grokking System Design Patterns treats each one directly.
• Practice two designs on paper. The document search system above, then a document permission system. Thirty minutes each, spoken aloud. Time the requirements step and keep it under five minutes.
• Prepare the companion rounds. The behavioral screen is covered in Top Harvey Behavioral Interview Questions (and How to Answer Them). The motivation question is covered in How to Answer: "Why Do You Want to Work at Harvey?".