Logging Traces for Evaluation

The trace log of the job ticket assistant makes four promises:

• Every request can be run again as it first ran, because the log records the input, the prompt version, the model and its settings.
• Every output is stored as the assistant returned it, before a dispatcher edits anything.
• Every later correction, by a dispatcher or a customer, is linked to the request it corrects.
• Every grade, from a judge or a person, is attached to the trace it grades.

It also makes two non-promises