decodeUtf8NoBomStrip function

String decodeUtf8NoBomStrip(
  1. Uint8List bytes
)

Decodes UTF-8 bytes to a Dart String without stripping leading U+FEFF. dart:convert's Utf8Decoder treats a leading BOM (U+FEFF / 0xEF 0xBB 0xBF) as a stream signature and drops it. Bridge strings are raw data, not streams.

Rather than hand-roll a per-byte code-point loop, use the fast native Utf8Decoder: count and strip any leading BOM byte-triples so the decoder never sees a BOM to drop, decode the remainder in one pass, then re-prepend exactly the U+FEFFs that were removed. Correct for any number of leading BOMs; the common (no-BOM) case is a single decoder call.

Implementation

String decodeUtf8NoBomStrip(Uint8List bytes) {
  if (bytes.isEmpty) return '';
  var bomBytes = 0;
  while (bomBytes + 3 <= bytes.length &&
      bytes[bomBytes] == 0xEF &&
      bytes[bomBytes + 1] == 0xBB &&
      bytes[bomBytes + 2] == 0xBF) {
    bomBytes += 3;
  }
  final decoded = utf8DecoderAllowMalformed.convert(bytes, bomBytes);
  if (bomBytes == 0) return decoded;
  return String.fromCharCode(0xFEFF) * (bomBytes ~/ 3) + decoded;
}