Grokking the Forward Deployed Engineer Interview
Vote

0% completed

​

Cost and Latency at Customer Scale

One claim costs the insurer about 2.3 cents in model use. At 1,200 claims a day, that is 27 dollars a day, or 810 dollars in a 30-day month. A claim with a scanned police report takes about 17 seconds, and the insurer's target is under an hour.

Neither number is a problem at today's volume, but either can become one if the design changes, the volume rises, or a person starts to wait for the answer.

Tokens and Price

Model use is priced by the amount of text the model reads and writes. A model splits text into pieces called tokens, about three quarters of an English word each

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%


Vote for new content