fuzzy_duplicate_detector 1.0.0 copy "fuzzy_duplicate_detector: ^1.0.0" to clipboard
fuzzy_duplicate_detector: ^1.0.0 copied to clipboard

A lightweight Dart/Flutter package for detecting fuzzy duplicates in Arabic and English text using phonetic matching, edit distance, and similarity scoring, with support for diacritics removal, transl [...]

Changelog #

All notable changes to fuzzy_duplicate_detector will be documented here. This project adheres to Semantic Versioning.


1.0.0 - 2026-06-02 #

Added #

  • ArabicSoundex — full phonetic encoding for Arabic names with Unicode normalization (diacritics, Alef variants, Teh Marbuta, Alef Maqsura, Tatweel, zero-width characters).
  • EnglishSoundex — standard Russell & Odell Soundex for Latin names with vowel-separator handling for improved accuracy.
  • Levenshtein — Unicode-safe edit-distance similarity using optimised two-row dynamic programming (O(m×n) time, O(min(m,n)) space).
  • JaroWinkler — prefix-weighted string similarity (standard 0.1 scaling factor, max 4-character prefix bonus).
  • TokenSetRatio — word-order-independent matching by sorting tokens before comparison; handles "Ali Mohamed" vs "Mohamed Ali".
  • Transliterator — bidirectional Arabic ↔ Latin phonetic transliteration for cross-script matching ("محمد" ↔ "Mohamed").
  • ArabicNormalizer — full Arabic text normalisation pipeline: diacritics, Alef variants, Teh Marbuta, Alef Maqsura, Tatweel, zero-width chars.
  • LatinNormalizer — Latin text normalisation: accent removal, punctuation stripping, case folding.
  • ScoreCombiner — configurable weighted fusion of all algorithm scores with optional transliteration bonus.
  • UnionFind — path-compressed, union-by-rank Disjoint Set Union for transitive duplicate grouping in O(α(n)) per operation.
  • GroupBuilder — pairwise O(n²) mode for small datasets; Soundex-blocked O(n·L + k²·L) mode for large datasets (≥ threshold).
  • FuzzyDedup — clean static-method public API: find(), compare(), findDuplicateItems(), findAsMap().
  • DedupConfig — fully configurable: threshold, algorithm weights, preprocessing toggles, clustering threshold; strict and lenient presets.
  • AlgorithmWeights — per-algorithm weight tuning with compile-time sum-to-1.0 assertion.
  • MatchResult — detailed per-pair result object with all raw scores.
  • DuplicateGroup — rich group model with canonical name, members list, average confidence, and all pairwise results.
  • Full unit-test suite (35+ tests across all modules).
  • Benchmark suite for 50-name and 1000-name datasets.
  • Complete example application with CRM use-case demonstration.
0
likes
140
points
23
downloads

Documentation

API reference

Publisher

unverified uploader

Weekly Downloads

A lightweight Dart/Flutter package for detecting fuzzy duplicates in Arabic and English text using phonetic matching, edit distance, and similarity scoring, with support for diacritics removal, transliteration, and word-order-independent comparison.

Repository (GitHub)
View/report issues

Topics

#text #nlp #fuzzy-matching #deduplication #arabic

License

MIT (license)

Dependencies

characters, collection

More

Packages that depend on fuzzy_duplicate_detector