InferenceModelSession class abstract
Session managing response generation from the model.
- Implementers
Constructors
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
Methods
-
addQueryChunk(
Message message) → Future< void> -
close(
) → Future< void> -
getResponse(
) → Future< String> -
getResponseAsync(
) → Stream< String> - Streams generated tokens.
-
getSessionMetrics(
) → SessionMetrics - Get session metrics including token usage and performance stats.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
sizeInTokens(
String text) → Future< int> -
stopGeneration(
) → Future< void> -
Stops the current generation, cancelling native decoding on the
.litertlmFFI path. Call this (orcancel()the response subscription) whenever a streamed response is superseded or no longer needed. -
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited