TraceExporter class abstract

Streams trial events to an external observability backend.

"Trace" here is OTel-style (a tree of spans for one trial), distinct from Transcript (the serialized record). A single trial produces:

  • one onTrialStart call
  • many onLLMCall and onToolCall interleaved
  • one onTrialEnd call

Implementations must be safe to call from concurrent trials. Each trial's events arrive in order but trials interleave with each other.

Implementers

Constructors

TraceExporter()

Properties

hashCode → int
The hash code for this object.
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

dispose() → Future<void>
Flush + release resources. Called once at the end of the run.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
onLLMCall({required Trial trial, required List<LLMMessage> requestMessages, required ModelConfig modelConfig, required ModelMessage? response, required Duration duration, Object? error}) → Future<void>
Called for each LLM call. response is null on errors.
onRunEnd({required String runName, required String suiteName, required Map<String, double> aggregateScores}) → Future<void>
Called when a dataset run completes. Implementations may use this hook to record run-level aggregate scores (pass@k, F1 …).
onToolCall({required Trial trial, required ToolCallRecord record}) → Future<void>
Called for each tool call.
onTrialEnd({required Trial trial, required Transcript transcript, required Outcome outcome, required List<Score> scores}) → Future<void>
Called when trial finishes with the final transcript, outcome, and scores. Implementations should attach all scores here.
onTrialStart(Trial trial, EvalTask task) → Future<void>
Called when a trial starts.
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited