Embeddings and Two-Tower Retrieval

Ids Carry No Meaning

Start with the inputs. A user is user_84302915. An item is item_5521. The model needs numbers that carry meaning. These ids carry none at all.

The simplest encoding makes it worse. One-hot at 100 million ids means a vector 100 million positions wide. One slot holds a 1. The rest hold zeros.

That is far too wide, and it still says nothing about similarity. Every id sits exactly as far from every other id as any other pair.

An embedding fixes both problems at once. It is a learned short vector, tens to hundreds of floats.