Embeddings and Two-Tower Retrieval

1. Ids Say Nothing

Meet your inputs. A user is user_84302915. An item is item_5521. The model needs numbers that carry meaning. These ids carry none at all.

The naive encoding makes it worse. One-hot at 100 million ids means a vector 100 million positions wide. One slot holds a 1. The rest hold zeros. That is astronomically wide, and it still says nothing about similarity: every id sits exactly as far from every other id as any other pair.

An embedding fixes both problems at once. It is a learned short vector, tens to hundreds of floats