twin_stream #

Simultaneous speech-to-text and audio file recording for Flutter.

twin_stream solves the notorious Android microphone lock issue — where using speech recognition (speech_to_text) and audio recording (record) simultaneously throws a microphone lock exception. This package provides a unified API that handles both operations in a single call, with intelligent platform-specific strategies.

✨ Features #

• 🎤 One API, dual output — Get both a transcript and an audio file from a single recording session
• 🤖 Android microphone lock handled — Automatically falls back to sequential mode when simultaneous mic access fails
• 🧠 True Android Simultaneous STT — Fully offline, true simultaneous STT via Vosk streaming audio bypass
• 🍎 True simultaneous mode on iOS/macOS — Leverages shared audio sessions
• 📡 Real-time state stream — Live transcript updates, sound levels, and recording duration
• ⚙️ Highly configurable — Sample rate, encoder, locale, STT options, and more
• 🛡️ Type-safe error handling — Custom exception hierarchy for precise error catching

📱 Platform Support #

Android Note: True simultaneous microphone access is restricted at the OS level on most modern versions natively. twin_stream provides two paths:

1. Provide a voskModelPath to use the embedded local AI streaming engine, enabling true offline simultaneous STT on all Android instances.
2. Skip Vosk, and the system relies on the stock Android SpeechRecognizer (will intelligently attempt simultaneous and gracefully fallback to sequential if the mic gets locked).

🚀 Quick Start #

Installation #

dependencies:
  twin_stream: ^0.1.0

Basic Usage #

import 'package:twin_stream/twin_stream.dart';

// Create and initialize
final controller = TwinStreamController();
await controller.initialize(
  config: TwinStreamConfig(localeId: 'en-US'),
);

// Listen for real-time updates
controller.stateStream.listen((state) {
  print('Live transcript: ${state.currentTranscript}');
  print('Sound level: ${state.soundLevel}');
});

// Start recording + speech recognition
await controller.start();

// ... user speaks ...

// Stop and get results
final result = await controller.stop();
print('Audio file: ${result.audioFilePath}');
print('Transcript: ${result.transcript}');
print('Confidence: ${result.confidence}');

// Clean up
controller.dispose();

Flutter Widget Example #

class RecorderPage extends StatefulWidget {
  @override
  State<RecorderPage> createState() => _RecorderPageState();
}

class _RecorderPageState extends State<RecorderPage> {
  final _controller = TwinStreamController();
  String _transcript = '';
  bool _isRecording = false;

  @override
  void initState() {
    super.initState();
    _init();
  }

  Future<void> _init() async {
    await _controller.initialize(
      config: TwinStreamConfig(localeId: 'en-US'),
    );

    _controller.stateStream.listen((state) {
      setState(() {
        _transcript = state.currentTranscript ?? '';
        _isRecording = state.status == TwinStreamStatus.recording;
      });
    });
  }

  Future<void> _toggleRecording() async {
    if (_isRecording) {
      final result = await _controller.stop();
      print('Saved to: ${result.audioFilePath}');
      print('Final: ${result.transcript}');
    } else {
      await _controller.start();
    }
  }

  @override
  void dispose() {
    _controller.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: [
            Text(_transcript.isEmpty ? 'Tap to record' : _transcript),
            const SizedBox(height: 20),
            FloatingActionButton(
              onPressed: _toggleRecording,
              child: Icon(_isRecording ? Icons.stop : Icons.mic),
            ),
          ],
        ),
      ),
    );
  }
}

⚙️ Configuration #

final config = TwinStreamConfig(
  // Audio settings
  sampleRate: 16000,          // Hz (16kHz optimal for STT)
  numChannels: 1,             // Mono recommended for STT
  bitRate: 128000,            // bps
  encoder: AudioEncoder.wav,  // WAV, AAC, Opus, FLAC, PCM

  // Speech recognition
  localeId: 'en-US',          // BCP-47 locale
  voskModelPath: null,        // Pass a directory path string to an extracted Vosk model for true Android simultaneous STT
  onDevice: false,            // Prefer on-device native recognition
  partialResults: true,       // Emit interim transcripts
  listenFor: Duration(seconds: 30),  // Max listen duration
  pauseFor: Duration(seconds: 3),    // Silence timeout

  // Android strategy
  androidStrategy: AndroidRecordingStrategy.simultaneousWithFallback,

  // File output
  outputDirectory: '/custom/dir',  // null = temp directory
  outputFileName: 'my_recording',  // null = timestamped name
);

Android Recording Strategies #

📊 State Stream #

The stateStream emits TwinStreamState objects with real-time session data:

controller.stateStream.listen((state) {
  state.status;             // TwinStreamStatus enum
  state.currentTranscript;  // Live partial transcript
  state.finalTranscript;    // Confirmed final transcript
  state.soundLevel;         // Mic level in dB
  state.audioFilePath;      // Output file path
  state.recordingDuration;  // Session duration
  state.isSimultaneous;     // True if running in simultaneous mode
  state.error;              // TwinStreamException if error occurred
});

🛡️ Error Handling #

try {
  await controller.initialize();
  await controller.start();
} on TwinStreamPermissionException catch (e) {
  // Microphone or speech recognition permission denied
  print('Permission needed: ${e.message}');
} on TwinStreamMicrophoneLockException catch (e) {
  // Android mic lock (only with simultaneousStrict strategy)
  print('Mic locked: ${e.message}');
} on TwinStreamInitializationException catch (e) {
  // STT unavailable or recorder setup failed
  print('Init failed: ${e.message}');
} on TwinStreamRecordingException catch (e) {
  // Error during active session
  print('Recording error: ${e.message}');
} on TwinStreamException catch (e) {
  // Catch-all for any TwinStream error
  print('Error: ${e.message}');
}

🔧 Platform Setup #

Android #

Add to android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.BLUETOOTH"/>
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN"/>
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT"/>

<!-- Required for Android SDK 30+ -->
<queries>
    <intent>
        <action android:name="android.speech.RecognitionService" />
    </intent>
</queries>

Requirements:

• minSdkVersion: 23
• compileSdkVersion: 31+
• Google app must be installed and enabled on the device

iOS #

Add to ios/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs microphone access for audio recording.</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>This app needs speech recognition to transcribe your voice.</string>

Requirements:

• iOS 12.0+

macOS #

Add to macos/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app needs microphone access for audio recording.</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>This app needs speech recognition to transcribe your voice.</string>

Add to macos/Runner/DebugProfile.entitlements and macos/Runner/Release.entitlements:

<key>com.apple.security.device.audio-input</key>
<true/>

Requirements:

• macOS 10.15+

🗣️ Language Support #

List available locales and let users choose:

final locales = await controller.availableLocales;
for (final locale in locales) {
  print('${locale.localeId}: ${locale.name}');
}

// Use a specific locale
await controller.initialize(
  config: TwinStreamConfig(localeId: 'hi-IN'), // Hindi
);

📋 API Reference #

TwinStreamController #

TwinStreamResult #

❓ FAQ #

Q: Why not just use record and speech_to_text separately? On Android, the microphone is a single hardware input. Using both packages simultaneously throws a microphone lock exception. twin_stream handles this automatically.

Q: Does simultaneous mode work on all Android devices? No. Most Android devices restrict concurrent microphone access. twin_stream uses simultaneousWithFallback by default, which gracefully handles this.

Q: What audio formats are supported? WAV (default), AAC (LC/ELD/HE), Opus, FLAC, and raw PCM. WAV is recommended for maximum compatibility.

Q: How does the Vosk model work for Android? By setting voskModelPath in your config to an extracted Vosk model directory located on the device, twin_stream will read precise PC16 byte chunks and push them directly to a local offline AI engine while simulatanously writing them to a wav file, entirely bypassing the Android mic lock!

Q: Can I use this for continuous/always-on listening? This package is designed for session-based recording (press to start, press to stop). The underlying speech_to_text platform APIs have timeout limits (typically 1 minute on iOS).

📄 License #

MIT License — see LICENSE for details.

🙏 Credits #

Built on top of these excellent packages:

• record by cow-level.ovh
• speech_to_text by csdcorp.com