utf16_safe_text #

Truncate and sanitize Dart strings without splitting UTF-16 surrogate pairs.

The crash this prevents #

A Dart String is a sequence of UTF-16 code units. Emoji and other characters outside the Basic Multilingual Plane are stored as a surrogate pair — two code units that only mean something together. substring counts units, not characters, so it can cut between the halves:

// 'Cem 👋' is 6 code units — the emoji is the last two.
'Cem 👋'.substring(0, 5); // 'Cem ' + half an emoji

Dart carries that string around without complaint. Skia does not. As soon as it reaches the text renderer:

Invalid argument(s): string is not well-formed UTF-16
(dart:ui _NativeParagraphBuilder.addText)

One user with an emoji in their display name is enough to crash every screen that truncates that name.

Usage #

import 'package:utf16_safe_text/utf16_safe_text.dart';

// Length-limit a string. Never splits a pair — the emoji is dropped whole.
safeTruncate('Cem 👋', 5); // 'Cem '
safeTruncate('Cem 👋', 6); // 'Cem 👋'

// Repair a string that is already malformed (database, API, another client).
stripLoneSurrogates('Cem \uD83D'); // 'Cem '

// Check before rendering.
isWellFormedUtf16('Cem 👋'); // true

Also available as extensions on String:

'Cem 👋'.truncateUtf16Safe(5);   // 'Cem '
'Cem \uD83D'.withoutLoneSurrogates; // 'Cem '
'Cem 👋'.isWellFormedUtf16;      // true

API #

Scope: pairs, not grapheme clusters #

This package guarantees well-formed UTF-16. It does not guarantee that user-perceived characters stay whole.

A family emoji like 👨‍👩‍👧 is several people joined by zero-width joiners. safeTruncate will not produce a malformed string from it — nothing crashes — but a cut can still leave you with fewer people than you started with. Likewise a combining accent can be separated from its base letter.

That is a deliberate boundary. Guarding against a renderer crash is cheap and belongs everywhere; iterating grapheme clusters is more expensive and is already solved by the characters package:

import 'package:characters/characters.dart';

'👨‍👩‍👧 hi'.characters.take(1).toString(); // '👨‍👩‍👧'

Use characters when you want "3 characters" to mean what a user thinks it means. Use this package when you need a hard code-unit ceiling — a database field limit, a protocol constraint — without risking malformed output.

Origin #

Extracted from ikiliya, a multiplayer word game where display names come from users and are truncated on nearly every screen.

License #

MIT