tokenize method

  1. @override
List<String> tokenize(
  1. String text
)
override

Tokenizes text into individual words/tokens

Implementation

@override
List<String> tokenize(String text) {
  return text.split(RegExp(r'\s+')).where((word) => word.isNotEmpty).toList();
}