you say "How do we know which server holds the connection to which user? We can introduce a software load balancer in front of our chat servers; that can map each UserID to a server to redirect the request."

Lets say there are 10 thousand users , how can we configure the reverse proxy to map these users to the chat server? Would it not be cumbersome? Will not the laoad balancer need to maintain these user info anywhere?