Embeddings and Two-Tower Retrieval

Ids Carry No Meaning

Start with the inputs. A user is user_84302915. An item is item_5521. The model needs numbers that carry meaning. These ids carry none at all.

The simplest encoding makes it worse. One-hot at 100 million ids means a vector 100 million positions wide. One slot holds a 1. The rest hold zeros.

A dense representation would be very large. Sparse storage avoids storing every zero, but one-hot encoding alone does not express learned similarity between ids.

An embedding solves both the width problem and the meaning problem at once