TextTokenizer class abstract

Interface for tokenization functionality

Implementers

Constructors

TextTokenizer.new()

Properties

hashCode → int
The hash code for this object.
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
supportsArabic → bool
Checks if the tokenizer supports Arabic text
no setter

Methods

noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
tokenize(String text) → List<String>
Tokenizes text into individual words/tokens
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited