Chunking and Indexing

Know the Vocabulary

This lesson uses the terms below many times. Read them once now, and come back here when a later section needs one.

• Ingestion: The background pipeline that loads documents into the retrieval system and keeps them current.
• Connector: A small service that reads documents from one source, such as the help center or the company wiki.
• Parsing: Turning a file, such as a web page or a PDF, into clean text. Good parsing keeps the headings, lists, and tables intact.
• Boilerplate: The menus, footers, and banners wrapped around a page's real content.