countTokens method

  1. @override
Future<int> countTokens({
  1. required int sessionId,
  2. required String text,
})
override

Exact count where the host has a tokenizer; throws LocalAiTokenizerUnavailable where it doesn't, so the caller decides whether to estimate.

Measured against sessionId. The native tokenizers are model-wide and ignore it; the web arm measures on that session, after any turn it is generating has finished.

Implementation

@override
Future<int> countTokens({
  required int sessionId,
  required String text,
}) async {
  calls.add('countTokens');
  countTokensSessionIds.add(sessionId);
  final error = countTokensError;
  if (error != null) throw error;
  return countTokensResult;
}