Grokking the Forward Deployed Engineer Interview
0% completed
Cost and Latency at Customer Scale
One claim costs the insurer about 2.3 cents in model use. At 1,200 claims a day, that is 27 dollars a day, or 810 dollars in a 30-day month. A claim with a scanned police report takes about 17 seconds, and the insurer's target is under an hour.
Neither number is a problem at today's volume, but either can become one if the design changes, the volume rises, or a person starts to wait for the answer.
Tokens and Price
Model use is priced by the amount of text the model reads and writes. A model splits text into pieces called tokens, about three quarters of an English word each
.....
.....
.....
Like the course? Get enrolled and start learning!
Reading Progress
0%