dart_bert_tokenizer 1.3.0 
dart_bert_tokenizer: ^1.3.0 copied to clipboard
Published 8 days ago • 
brodykim.work
brodykim.workA pure Dart BERT WordPiece tokenizer with Hugging Face tokenizer.json loading, pre-tokenized alignment, dynamic AddedTokens, Unicode offsets, and verified model fixtures.
6 versions
Stable versions of dart_bert_tokenizer
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.3.0 | 3.10 | 8 days ago |  |  | |
| 1.2.0 | 3.10 | 8 days ago | | | |
| 1.1.0 | 3.10 | 8 days ago | | ||
| 1.0.2 | 3.10 | 9 months ago | | ||
| 1.0.1 | 3.10 | 9 months ago | | ||
| 1.0.0 | 3.10 | 9 months ago | |
Metadata
2
likes
160
points
526
downloads
Documentation
Publisher
Weekly Downloads
Metadata
A pure Dart BERT WordPiece tokenizer with Hugging Face tokenizer.json loading, pre-tokenized alignment, dynamic AddedTokens, Unicode offsets, and verified model fixtures.
Repository (GitHub)
View/report issues
Topics
#nlp #bert #tokenizer #machine-learning #wordpiece
License
MIT (license)