TextPreprocessor class
Converts text to numeric tensors for TFLite model input.
Supports character-level encoding (each character → integer index) which is used by typo detection and text classification models.
Constructors
- TextPreprocessor({String vocab = defaultVocab, int maxLength = 64})
- Creates a preprocessor with the given vocabulary and max sequence length.
Properties
- hashCode → int
-
The hash code for this object.
no setterinherited
- maxLength → int
-
The maximum sequence length.
no setter
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- vocabSize → int
-
The size of the vocabulary (including padding and unknown tokens).
no setter
Methods
-
encode(
String text) → List< int> - Encodes a string into a fixed-length integer sequence.
-
encodeAsFloat32(
String text) → Float32List - Encodes text and returns it as a Float32List suitable for TFLite input.
-
encodeAsTensor(
String text) → List< List< double> > -
Encodes text into a shaped tensor:
1, maxLengthfor single inference. -
encodeBatch(
List< String> texts) → List<List< double> > -
Batch-encodes multiple texts into a
batchSize, maxLengthtensor. -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited
Constants
- defaultVocab → const String
- Default character vocabulary: ASCII printable characters.