flutter_baidu_speech_tts #

English | 中文

A Flutter plugin for Baidu Text-to-Speech (TTS), supporting online, offline, and mixed (MIX) synthesis.

Platform support:

• Android: Online / Offline / Mixed synthesis. Supports accessToken, apiKey + secretKey, and iamKey authentication.
• iOS: Online / Offline / Mixed synthesis. Supports accessToken, apiKey + secretKey, and iamKey authentication. Device only (SDK static library contains only the arm64 device architecture; simulator architectures are excluded in the podspec).
• OHOS (HarmonyOS): Online / Offline / Mixed synthesis. Supports accessToken and apiKey + secretKey. iamKey and offlineOverwriteAssets are not applicable; if passed, they will be listed in the ignoredParams field of the initialize return value. Offline models are loaded directly from the path under context.resourceDir (mapped to resources/resfile/), with no copying.

All three platforms support "plugin self-playback PCM + text highlight following" (initialize with playbackMode: BaiduTtsPlaybackMode.plugin), see Text Highlight Following.

1. Prerequisites #

Create an app on the Baidu AI Speech Platform to obtain:

• Online synthesis: apiKey + secretKey (or accessToken)
• Offline synthesis: additionally requires appId + authSn
• Offline model files (.dat, 8–16 MB each): text model + voice model. The plugin does not bundle models — you must download them separately and place them in your project.

Note: authSn is bound to the app's package name / BundleId. Each platform (Android / iOS / OHOS) must register its own app with independent credentials. The recommended approach is to dispatch credentials per-platform on the Dart side (see example lib/utils/tts_config.dart).

2. Add Dependency #

In pubspec.yaml:

dependencies:
  flutter_baidu_speech_tts: ^1.0.4

Then run flutter pub get. Native registration on all three platforms is auto-generated by the Flutter toolchain — no manual setup required:

• Android: GeneratedPluginRegistrant.java registers com.baidu.flutter.tts.FlutterBaiduTtsPlugin
• iOS: pod install generates the flutter_baidu_speech_tts pod
• OHOS: GeneratedPluginRegistrant.ets registers FlutterBaiduTtsPlugin and injects com.baidu.tts_*.har into entry dependencies

Core Features #

One codebase · Three platforms · Three synthesis modes — Full coverage for Android / iOS / HarmonyOS (OHOS), with online, offline, and mixed modes freely switchable, and three authentication methods for flexible integration.

Six Out-of-the-Box Capabilities #

Feature Demo #

Place screenshots in the same directory as this README; images render automatically.

Before Init — Configuration page for entering TTS credentials

Init Success — Engine loaded; code 0 means success

Synthesizing / Playing — Text is being converted to speech and played back

3. Quick Start #

final tts = FlutterBaiduTts();

// typedEvents is a broadcast stream; subscribers must cancel their own subscriptions.
final sub = tts.typedEvents.listen((BaiduTtsEvent e) {
  debugPrint('$e');
  // Synthesis data callback: e.event == 'SYNTHESIZE_DATA_ARRIVED', PCM data in e.audioData
});

final init = await tts.initializeWithConfig(const BaiduTtsConfig(
  apiKey: 'ak',
  secretKey: 'sk',
));
if (init.isSuccess) {
  await tts.speakText('Hello, Baidu speech synthesis');
}

// On page dispose
await sub.cancel();
await tts.releaseTts();

Credential & Parameter Management #

Refer to lib/utils/tts_config.dart: dispatch credentials per-platform via Platform.isAndroid / isIOS, then produce a unified BaiduTtsConfig:

static BaiduTtsConfig buildInitConfig() {
  return BaiduTtsConfig(
    apiKey: _apiKey,
    secretKey: _secretKey,
    onlineSpeaker: '4100',
    onlineTimeoutMs: 2000,
    enableOffline: true,
    // The following 4 fields are only needed for offline synthesis
    appId: _appId,
    authSn: _authSn,
    offlineTextModelAsset: 'bd_etts_common_text_txt_all_..._v6.0.0_20240731.dat',
    offlineSpeechModelAsset: 'bd_etts_common_speech_duxiaomei_..._20251031153737.dat',
  );
}

Key points:

• offlineTextModelAsset / offlineSpeechModelAsset take file names, not paths; the plugin resolves them per-platform.
• If models are downloaded to disk yourself, pass absolute paths via offlineTextModelPath / offlineSpeechModelPath — paths take priority over asset names.
• Do not commit real credentials to public repositories.

Full Call Flow #

final FlutterBaiduTts _tts = FlutterBaiduTts();

// 1) Subscribe to events first (broadcast stream, can listen multiple times, cancel yourself)
_sub = _tts.typedEvents.listen((BaiduTtsEvent e) {
  // e.event == 'SYNTHESIZE_DATA_ARRIVED' → e.audioData is a PCM chunk
});

// 2) Initialize
final init = await _tts.initializeWithConfig(TtsConfig.buildInitConfig());
if (!init.isSuccess) {
  // init.code / init.message; if offline enabled, also init.offlineCode / offlineMessage
}

// 3) Synthesize & play / synthesize only
await _tts.speakText(text, mode: BaiduTtsMode.offline); // online / offline / mix, default mix
await _tts.synthesizeText(text);

// 4) Control & release
await _tts.pauseTts();
await _tts.resumeTts();
await _tts.stopTts();
await _sub.cancel();
await _tts.releaseTts();

getCuid() returns the SDK device fingerprint, used to apply for offline authorization on the Baidu platform. It typically only has a value after initializeWithConfig, so you need to fetch it again after initialization completes.

Text Highlight Following #

BaiduTtsHighlightController advances the "spoken character count" based on the actual playback position during synthesis. Combined with BaiduTtsHighlightTextEditingController, it can directly color text inside a TextField — no separate read-only display area needed. Behavior is consistent across Android / iOS / OHOS.

Prerequisite: initialize with playbackMode: BaiduTtsPlaybackMode.plugin (plugin plays PCM itself), and set pcmSampleRate to the actual sample rate of the offline voice model (default 16000). When using the SDK's built-in player (BaiduTtsPlaybackMode.sdk), the real playback position is unavailable and can only be estimated by clock — hasRealPlaybackPosition will be false.

final tts = FlutterBaiduTts();
final highlight = BaiduTtsHighlightController();
final textController = BaiduTtsHighlightTextEditingController(
  highlight: highlight,
  text: 'Long text to be spoken...',
);

highlight.addListener(() => setState(() {}));

await tts.initializeWithConfig(BaiduTtsConfig(
  apiKey: 'ak',
  secretKey: 'sk',
  playbackMode: BaiduTtsPlaybackMode.plugin, // required for highlight
  pcmSampleRate: 16000,                       // match the voice model .dat sample rate
));

// Speak — highlight follows automatically
await highlight.speak(textController.text, mode: BaiduTtsMode.mix);
await highlight.pause();
await highlight.resume();
await highlight.stop();

// Use directly in your UI
TextField(controller: textController);

// On page dispose: dispose the text controller first, then the highlight controller
textController.dispose();
highlight.dispose();

Controller-exposed state (all notified via ChangeNotifier):

• text / readLength / synthLength: original text, spoken character count, synthesized character count
• progress: speaking progress 0–1
• isSpeaking / isPaused / hasRealPlaybackPosition
• lastError: the most recent SYNTHESIZE_ERROR event

BaiduTtsHighlightOptions lets you tune breakCharacters (sentence-break characters, default 。！？；\n\r.!?;), maxRequestUnits (max text units per request, default 900, matching the engine's 1024-byte limit), tickInterval (interpolation refresh interval, default 16ms), and defaultCharsPerSecond (fallback speech rate before first audio chunk arrives).

Implementation & trade-offs:

• Per-sentence splitting: Requests are split only at sentence-ending punctuation or line breaks — one sentence per request. Sentence boundaries naturally have pauses, making seams inaudible, while each independent request yields a precise anchor point. Error never accumulates across sentences. Only when a single sentence exceeds the engine limit does it hard-split at shorter pauses like commas.
• Intra-sentence mapping from engine alignment: Each PCM chunk carries synthesis progress (BaiduTtsEvent.audioProgress + progressUnit — iOS reports character count, Android/OHOS report GBK byte offset). This builds an "audio frame → character" lookup table, queried by playback position; between two position reports, interpolation runs at tickInterval. Highlight only advances forward, never backward.
• Trade-off: After splitting, each segment is independently re-synthesized — the engine redoes prosody planning, and you may hear timbre/tone changes at seams (especially noticeable with am-tac-csubgan16k voice models).
• During playback, do not mix tts.speakText / pauseTts / stopTts — route everything through the controller's speak / pause / resume / stop.
• Editing the input field during playback stops highlighting (indices are calculated against the original text at speak time; once content changes, indices become misaligned).

Plugin self-playback mode event differences: adds PLAY_POSITION (positionMs / durationMs, from the hardware playback head); SYNTHESIZE_DATA_ARRIVED no longer carries audioData (saving one copy per chunk) but still includes audioBytes and sampleRate.

Error Handling Convention #

Every method returns a result object shaped like {code, message, ...}. On failure, code != 0 — no PlatformException is thrown. Do not use try/catch to determine success; use isSuccess. Reserved negative error codes:

• -1: General error (missing params, not initialized, SDK internal error, etc.)
• -2: initialize in progress (concurrent call)

Other non-zero values come from Baidu SDK error codes (Android: getDetailCode(), iOS: NSError.code). When initialize fails, the return value also carries offlineCode / offlineMessage (offline engine load result) and paramErrors (details of params rejected by the SDK).

To access raw Map return values, use FlutterBaiduTtsPlatform.instance directly.

4. Offline Models #

Offline models (.dat files, 8–16 MB each, ~56 MB total) are Baidu proprietary licensed files and are not distributed with the plugin. For offline synthesis, integrators must obtain model files from the Baidu AI Speech Platform, place them in the appropriate native resource directory of their project, and reference them by file name (not path) via offlineTextModelAsset / offlineSpeechModelAsset:

await tts.initializeWithConfig(BaiduTtsConfig(
  apiKey: 'ak',
  secretKey: 'sk',
  appId: 'appId',
  authSn: 'authSn',
  enableOffline: true,
  offlineTextModelAsset: 'bd_etts_common_text_txt_all_mand_eng_middle_big_v6.0.0_20240731.dat',
  offlineSpeechModelAsset:
      'bd_etts_common_speech_duxiaoyu_mand_eng_high_am-tac-csubgan16k_v4.9.0_20240918_20251031153737.dat',
));

Placement directories per platform:

• Android: android/app/src/main/assets/ (copied to filesDir on first initialize; offlineOverwriteAssets: true forces overwrite)
• iOS: Add to Xcode Runner target (enters Bundle.main)
• OHOS: entry/src/main/resources/resfile/ (resolved to a path under context.resourceDir, no copying)

You can also download models to disk yourself and pass absolute paths via offlineTextModelPath / offlineSpeechModelPath — paths take priority over file names. If a path does not exist, initialize returns failure immediately — no silent fallback.

5. Android Integration #

Minimal changes — the android/ directory is essentially template defaults:

• Permissions: The plugin's own manifest already declares INTERNET and ACCESS_NETWORK_STATE, which merge into the host. The host does not need to redeclare them. Optionally add READ_PHONE_STATE (makes cuid more stable; Android 10+ can no longer obtain IMEI, only declare if needed) and READ_EXTERNAL_STORAGE (if models are placed outside the sandbox).
• applicationId: Must match the package name registered on the Baidu platform.
• minSdk / targetSdk / ndkVersion: Use flutter.* defaults. The SDK's jar (in android/libs/) and .so files (in android/src/main/jniLibs/, covering arm64-v8a / armeabi-v7a / x86 / x86_64) are already packaged in the plugin AAR — no extra repository or abiFilters needed.
• ProGuard: Rules provided by the plugin's consumerProguardFiles (android/consumer-rules.pro); enabling minifyEnabled requires no extra configuration.

Offline model placement:

android/app/src/main/assets/
  bd_etts_common_text_txt_all_mand_eng_middle_big_v6.0.0_20240731.dat   # Text model
  bd_etts_common_speech_duxiaomei_..._20251031153737.dat                # Voice model

6. iOS Integration #

Device only — The Baidu iOS static library contains only the arm64 device slice, and the podspec sets EXCLUDED_ARCHS[sdk=iphonesimulator*] to exclude simulators.

1. Static library libBDSpeechTTSBaseKit.a (~239 MB): Not published with the plugin. During pod install, the podspec auto-downloads it to the plugin's ios/Libs/ (override the download URL with the FLUTTER_BAIDU_TTS_IOS_LIB_URL environment variable). Linking is handled by Pods-Runner.xcconfig: OTHER_LDFLAGS includes -ObjC -l"BDSpeechTTSBaseKit", and LIBRARY_SEARCH_PATHS points to .symlinks/plugins/flutter_baidu_speech_tts/ios/Libs. If download fails or you prefer manual management, copy BDSClientLib/libBDSpeechTTSBaseKit.a from the Baidu iOS TTS SDK package (BDSpeechClientSDK_TTS) to ios/Libs/.

2. Add offline models to the Runner target's Resources (enters Bundle.main). To avoid duplicate storage, this project references the Android assets directory directly: in ios/Runner.xcodeproj/project.pbxproj, each .dat file's path is set to ../android/app/src/main/assets/xxx.dat and added to PBXResourcesBuildPhase. In Xcode, drag them in and check the Runner target.

   Model path resolution order: offlineXxxModelPath absolute path → Bundle.main → sandbox Documents/. If not found or loadOfflineEngine fails, initialize returns failure immediately — no silent fallback to online.

3. ios/Runner/Info.plist: This project adds NSLocalNetworkUsageDescription ("This app needs to access the local network to support relevant features"). TTS only plays audio — no microphone permission needed.

4. Deployment target: IPHONEOS_DEPLOYMENT_TARGET = 12.0. The Podfile does not explicitly specify platform, using the Flutter default.

5. Audio session: Managed by the SDK itself (the plugin sets the category to playback). If the host needs to manage AVAudioSession itself, override it after initialize.

Note: Do not manually link static libraries outside the project (e.g., absolute paths like ../../BDSpeechClientSDK_.../BDSClientLib/libBDSpeechTTSBaseKit.a). These are local debugging leftovers — the plugin's pod already handles linking the same library. Using such paths will cause build failures on other machines or directories. If such File References exist in your project, remove them.

7. OHOS (HarmonyOS) Integration #

1. You must declare network permissions yourself. The HAR's module.json5 does not participate in the final build, so its permissions do not merge into the host. In ohos/entry/src/main/module.json5:

"requestPermissions": [
  { "name": "ohos.permission.INTERNET" },
  { "name": "ohos.permission.GET_NETWORK_INFO" }
]

Under products, configure: "buildOption": {
            "strictMode": {
            "useNormalizedOHMUrl": true
          }

Missing INTERNET causes online authorization (PARAM_LICENSE_URL) to fail during initialization, making synthesis completely unusable. If ohosTest has networked test cases, declare it there too.

2. Place offline models in entry/src/main/resources/resfile/ and pass file names; the plugin resolves them to paths under context.resourceDir — no copying.

3. Dependencies: com.baidu.tts_*.har + authbaselibrary.har are injected from the plugin's ohos/libs/ into entry by the Flutter toolchain (see ohos/entry/oh-package-lock.json5) — no manual oh-package.json5 dependencies needed.

4. SDK version: The example project uses compatibleSdkVersion 5.0.4(16), runtimeOS HarmonyOS (see ohos/build-profile.json5).

5. Limitations: iamKey and offlineOverwriteAssets are not supported; if passed, they are listed in the ignoredParams field of the initialize return value. The offline authorization URL is fixed to https://upl.baidu.com/auth and cannot be configured.

8. Running the Example #

The example's credentials are centralized in example/lib/utils/tts_config.dart (TtsConfig), split into Android / iOS / OHOS groups. Before running, replace apiKey / secretKey / appId / authSn with your own credentials:

cd example
flutter run               # Android
flutter run -d <device>   # iOS (simulator not supported)

Note: These credentials are currently plaintext constants, for local example runs only. Do not commit real credentials to public repositories.

9. Integration Checklist #

• ❌ Add dependency in pubspec.yaml, run flutter pub get
• ❌ Package name / BundleId matches Baidu platform registration on all three platforms; appId / authSn are platform-specific
• ❌ Android models placed in android/app/src/main/assets/
• ❌ iOS models added to Runner target Resources; running on device; after pod install, confirm ios/Libs/libBDSpeechTTSBaseKit.a exists
• ❌ OHOS models placed in entry/src/main/resources/resfile/; module.json5 declares ohos.permission.INTERNET
• ❌ Subscribe to typedEvents before calling initializeWithConfig
• ❌ Use result.isSuccess to check success; for offline failures, check offlineCode / offlineMessage
• ❌ For highlight following: initialize with playbackMode: BaiduTtsPlaybackMode.plugin, set pcmSampleRate to match the voice model's sample rate, and route all playback through BaiduTtsHighlightController
• ❌ On page dispose: sub.cancel() + releaseTts()

10. FAQ #

• initialize returns code != 0: Check message and paramErrors (specific params rejected by the SDK). For offline issues, check offlineCode / offlineMessage.
• Offline doesn't work but online is fine: Model file name is misspelled, models not placed in the correct resource directory, or appId / authSn missing. With mode: BaiduTtsMode.offline, there is no online fallback; only mix falls back from online to offline on failure.
• iOS simulator reports architecture error: Expected behavior — the static library has no simulator slice; device only.
• OHOS synthesis is silent or init fails: First check that the entry module declares ohos.permission.INTERNET.
• getCuid() returns empty: Call initializeWithConfig first, then fetch.
• For iOS, if pod install fails, refer to: https://cloud.baidu.com/doc/SPEECH/s/wltwwnvc9#5-sdk%E9%9B%86%E6%88%90 — follow the official guide to import resources into the project.

11. Known Limitations #

• The iOS static library is 239 MB, exceeding pub.flutter-io.cn's 100 MB single-package limit, so it is not distributed with the package. It is auto-downloaded during pod install (override with the FLUTTER_BAIDU_TTS_IOS_LIB_URL environment variable).
• iOS supports device only (the static library has no simulator slice).
• When upgrading the native SDK, re-verify its transitive dependencies: on the Android side, there is no dependency metadata after unpacking; the OkHttp used internally by the SDK is explicitly declared in android/build.gradle.
• The OHOS offline authorization URL is currently fixed to https://upl.baidu.com/auth and cannot be configured.
• On the Android side, the SDK's loadAudioPlayer() is not called; playback uses the SDK's default player.
• Text highlight following splits text into multiple requests per sentence — you may hear timbre/tone changes at seams. This is the trade-off for obtaining precise anchor points (see "Text Highlight Following").
• Plugin self-playback (BaiduTtsPlaybackMode.plugin) sample rate is declared by the caller via pcmSampleRate, not taken from the SDK-reported audio format (the iOS SDK reports 16 kHz as 8 kHz; building the playback format from that would drop pitch by an octave and double the duration). When using an 8 kHz voice model, pass 8000 accordingly.

中文文档