EnglishTextTokenizer class

English text tokenizer implementation

Implemented types

Constructors

EnglishTextTokenizer.new()

Properties

hashCode → int
The hash code for this object.
no setterinherited
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
supportsArabic → bool
Checks if the tokenizer supports Arabic text
no setteroverride

Methods

noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
tokenize(String text) → List<String>
Tokenizes text into individual words/tokens
override
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited