LLM Dart Library #

A modular Dart library for AI provider interactions. This library provides a unified interface for interacting with different AI providers using Dio for HTTP requests.

🧠 Full access to model thinking processes - llm_dart provides direct access to the reasoning and thought processes of supported AI models (Claude, DeepSeek, Gemini), giving you unprecedented insight into how AI models arrive at their conclusions.

🚀 Quick Navigation #

Features #

• Multi-provider support: OpenAI, Anthropic (Claude), Google (Gemini), DeepSeek, Groq, Ollama, xAI (Grok), ElevenLabs
• 🧠 Thinking process support: Access to model reasoning and thought processes (Claude, DeepSeek, Gemini)
• 🎵 Unified audio capabilities: Text-to-speech, speech-to-text, and audio processing with feature discovery
• 🖼️ Image generation & processing: DALL-E integration, image editing, variations, and multi-modal support
• 📁 File management: Unified file operations across providers (OpenAI, Anthropic)
• 🏭 Type-safe capability building: Compile-time type safety with capability factory methods
• Unified API: Consistent interface across all providers with capability-based design
• Builder pattern: Fluent API for easy configuration and provider setup
• Streaming support: Real-time response streaming with thinking process access
• Tool calling: Advanced function calling with enhanced patterns
• Structured output: JSON schema support with validation
• Error handling: Comprehensive error types with graceful degradation
• Type safety: Full Dart type safety with modular architecture
• MCP Integration: Model Context Protocol support for external tool connections

Supported Providers #

• 🧠 Thinking Process Support: Access to model's reasoning and thought processes
• 🎵 Audio Support: Text-to-speech, speech-to-text, and audio processing
• 🖼️ Image Support: Image generation, editing, and multi-modal processing
• 📁 File Support: File upload, management, and processing capabilities

Installation #

Add this to your pubspec.yaml:

dependencies:
  llm_dart: ^0.4.0

Then run:

dart pub get

Or install directly using:

dart pub add llm_dart

Quick Start #

Basic Usage #

import 'package:llm_dart/llm_dart.dart';

void main() async {
  // Method 1: Using the new ai() builder with provider methods
  final provider = await ai()
      .openai()
      .apiKey('your-api-key')
      .model('gpt-4')
      .temperature(0.7)
      .build();

  // Method 2: Using provider() with string ID (extensible)
  final provider2 = await ai()
      .provider('openai')
      .apiKey('your-api-key')
      .model('gpt-4')
      .temperature(0.7)
      .build();

  // Method 3: Using convenience function
  final directProvider = await createProvider(
    providerId: 'openai',
    apiKey: 'your-api-key',
    model: 'gpt-4',
    temperature: 0.7,
  );

  // Simple chat
  final messages = [ChatMessage.user('Hello, world!')];
  final response = await provider.chat(messages);
  print(response.text);

  // Access thinking process (for supported models)
  if (response.thinking != null) {
    print('Model thinking: ${response.thinking}');
  }
}

Streaming with DeepSeek Reasoning #

import 'dart:io';
import 'package:llm_dart/llm_dart.dart';

// Create DeepSeek provider for streaming with thinking
final provider = await ai()
    .deepseek()
    .apiKey('your-deepseek-key')
    .model('deepseek-reasoner')
    .temperature(0.7)
    .build();

final messages = [ChatMessage.user('What is 15 + 27? Show your work.')];

// Stream with real-time thinking process
await for (final event in provider.chatStream(messages)) {
  switch (event) {
    case ThinkingDeltaEvent(delta: final delta):
      // Show AI's thinking process in gray
      stdout.write('\x1B[90m$delta\x1B[0m');
      break;
    case TextDeltaEvent(delta: final delta):
      // Show final answer
      stdout.write(delta);
      break;
    case CompletionEvent(response: final response):
      print('\n✅ Completed');
      if (response.usage != null) {
        print('Tokens: ${response.usage!.totalTokens}');
      }
      break;
    case ErrorEvent(error: final error):
      print('Error: $error');
      break;
  }
}

🧠 Thinking Process Access #

Access the model's internal reasoning and thought processes:

// Claude with thinking
final claudeProvider = await ai()
    .anthropic()
    .apiKey('your-anthropic-key')
    .model('claude-sonnet-4-20250514')
    .build();

final messages = [
  ChatMessage.user('Solve this step by step: What is 15% of 240?')
];

final response = await claudeProvider.chat(messages);

// Access the final answer
print('Answer: ${response.text}');

// Access the thinking process
if (response.thinking != null) {
  print('Claude\'s thinking process:');
  print(response.thinking);
}

// DeepSeek with reasoning
final deepseekProvider = await ai()
    .deepseek()
    .apiKey('your-deepseek-key')
    .model('deepseek-reasoner')
    .temperature(0.7)
    .build();

final reasoningResponse = await deepseekProvider.chat(messages);
print('DeepSeek reasoning: ${reasoningResponse.thinking}');

Tool Calling #

final tools = [
  Tool.function(
    name: 'get_weather',
    description: 'Get weather for a location',
    parameters: ParametersSchema(
      schemaType: 'object',
      properties: {
        'location': ParameterProperty(
          propertyType: 'string',
          description: 'City name',
        ),
      },
      required: ['location'],
    ),
  ),
];

final response = await provider.chatWithTools(messages, tools);
if (response.toolCalls != null) {
  for (final call in response.toolCalls!) {
    print('Tool: ${call.function.name}');
    print('Args: ${call.function.arguments}');
  }
}

Provider Examples #

OpenAI #

final provider = await createProvider(
  providerId: 'openai',
  apiKey: 'sk-...',
  model: 'gpt-4',
  temperature: 0.7,
  extensions: {'reasoningEffort': 'medium'}, // For reasoning models
);

Anthropic (with Thinking Process) #

final provider = await ai()
    .anthropic()
    .apiKey('sk-ant-...')
    .model('claude-sonnet-4-20250514')
    .build();

final response = await provider.chat([
  ChatMessage.user('Explain quantum computing step by step')
]);

// Access Claude's thinking process
print('Final answer: ${response.text}');
if (response.thinking != null) {
  print('Claude\'s reasoning: ${response.thinking}');
}

DeepSeek (with Reasoning) #

final provider = await ai()
    .deepseek()
    .apiKey('your-deepseek-key')
    .model('deepseek-reasoner')
    .build();

final response = await provider.chat([
  ChatMessage.user('Solve this logic puzzle step by step')
]);

// Access DeepSeek's reasoning process
print('Solution: ${response.text}');
if (response.thinking != null) {
  print('DeepSeek\'s reasoning: ${response.thinking}');
}

Ollama #

final provider = ollama(
  baseUrl: 'http://localhost:11434',
  model: 'llama3.2',
  // No API key needed for local Ollama
);

ElevenLabs (Audio Processing) #

// Use buildAudio() for type-safe audio capability building
final audioProvider = await ai()
    .elevenlabs()
    .apiKey('your-elevenlabs-key')
    .voiceId('JBFqnCBsd6RMkjVDRZzb') // George voice
    .stability(0.7)
    .similarityBoost(0.9)
    .style(0.1)
    .buildAudio(); // Type-safe audio capability building

// Direct usage without type casting
final features = audioProvider.supportedFeatures;
print('Supports TTS: ${features.contains(AudioFeature.textToSpeech)}');

// Text to speech with advanced options
final ttsResponse = await audioProvider.textToSpeech(TTSRequest(
  text: 'Hello world! This is ElevenLabs speaking.',
  voice: 'JBFqnCBsd6RMkjVDRZzb',
  model: 'eleven_multilingual_v2',
  format: 'mp3_44100_128',
  includeTimestamps: true,
));
await File('output.mp3').writeAsBytes(ttsResponse.audioData);

// Speech to text (if supported)
if (features.contains(AudioFeature.speechToText)) {
  final audioData = await File('input.mp3').readAsBytes();
  final sttResponse = await audioProvider.speechToText(
    STTRequest.fromAudio(audioData, model: 'scribe_v1')
  );
  print(sttResponse.text);
}

// Convenience methods
final quickSpeech = await audioProvider.speech('Quick TTS');
final quickTranscription = await audioProvider.transcribeFile('audio.mp3');

Error Handling #

try {
  final response = await provider.chatWithTools(messages, null);
  print(response.text);
} on AuthError catch (e) {
  print('Authentication failed: $e');
} on ProviderError catch (e) {
  print('Provider error: $e');
} on HttpError catch (e) {
  print('Network error: $e');
} catch (e) {
  print('Unexpected error: $e');
}

Architecture #

Capability-Based Design #

The library uses a capability-based interface design instead of monolithic "god interfaces":

// Core capabilities
abstract class ChatCapability {
  Future<ChatResponse> chat(List<ChatMessage> messages);
  Stream<ChatStreamEvent> chatStream(List<ChatMessage> messages);
}

abstract class EmbeddingCapability {
  Future<List<List<double>>> embed(List<String> input);
}

// Providers implement only the capabilities they support
class OpenAIProvider implements ChatCapability, EmbeddingCapability {
  // Implementation
}

Type-Safe Capability Building #

The library provides capability factory methods for compile-time type safety:

// Old approach - runtime type casting
final provider = await ai().openai().apiKey(apiKey).build();
if (provider is! AudioCapability) {
  throw Exception('Audio not supported');
}
final audioProvider = provider as AudioCapability; // Runtime cast!

// New approach - compile-time type safety
final audioProvider = await ai().openai().apiKey(apiKey).buildAudio();
// Direct usage without type casting - guaranteed AudioCapability!

// Available factory methods:
final audioProvider = await ai().openai().buildAudio();
final imageProvider = await ai().openai().buildImageGeneration();
final embeddingProvider = await ai().openai().buildEmbedding();
final fileProvider = await ai().openai().buildFileManagement();
final moderationProvider = await ai().openai().buildModeration();
final assistantProvider = await ai().openai().buildAssistant();
final modelProvider = await ai().openai().buildModelListing();

// Clear error messages for unsupported capabilities
try {
  final audioProvider = await ai().groq().buildAudio(); // Groq doesn't support audio
} catch (e) {
  print(e); // UnsupportedCapabilityError: Provider "groq" does not support audio capabilities. Supported providers: OpenAI, ElevenLabs
}

Provider Registry #

The library includes an extensible provider registry system:

// Check available providers
final providers = LLMProviderRegistry.getRegisteredProviders();
print('Available: $providers'); // ['openai', 'anthropic', ...]

// Check capabilities
final supportsChat = LLMProviderRegistry.supportsCapability('openai', LLMCapability.chat);
print('OpenAI supports chat: $supportsChat'); // true

// Create providers dynamically
final provider = LLMProviderRegistry.createProvider('openai', config);

Custom Providers #

You can register custom providers:

// Create a custom provider factory
class MyCustomProviderFactory implements LLMProviderFactory<ChatCapability> {
  @override
  String get providerId => 'my_custom';

  @override
  Set<LLMCapability> get supportedCapabilities => {LLMCapability.chat};

  @override
  ChatCapability create(LLMConfig config) => MyCustomProvider(config);

  // ... other methods
}

// Register it
LLMProviderRegistry.register(MyCustomProviderFactory());

// Use it
final provider = await ai().provider('my_custom').build();

Configuration #

All providers support common configuration options:

• apiKey: API key for authentication
• baseUrl: Custom API endpoint
• model: Model name to use
• temperature: Sampling temperature (0.0-1.0)
• maxTokens: Maximum tokens to generate
• systemPrompt: System message
• timeout: Request timeout
• topP, topK: Sampling parameters

Provider-Specific Extensions #

Use the extension system for provider-specific features:

final provider = await ai()
    .openai()
    .apiKey('your-key')
    .model('gpt-4')
    .reasoningEffort(ReasoningEffort.high)  // OpenAI-specific
    .extension('voice', 'alloy')           // OpenAI TTS voice
    .build();

Examples #

See the example directory for comprehensive usage examples organized by learning path:

🟢 Getting Started #

Perfect for first-time users

• quick_start.dart - Quick experience with multiple providers
• provider_comparison.dart - Compare providers and choose the right one
• basic_configuration.dart - Essential configuration patterns

🟡 Core Features #

Master the essential functionality

• chat_basics.dart - Foundation of all AI interactions
• streaming_chat.dart - Real-time streaming responses
• tool_calling.dart - Function calling capabilities
• enhanced_tool_calling.dart - Advanced tool calling patterns
• structured_output.dart - JSON schema and validation
• error_handling.dart - Production-ready error handling
• capability_factory_methods.dart - Type-safe capability building

🔴 Advanced Features #

Cutting-edge AI capabilities

• reasoning_models.dart - 🧠 AI thinking processes and reasoning
• multi_modal.dart - Images, audio, and file processing
• custom_providers.dart - Build your own AI provider
• performance_optimization.dart - Production optimization techniques

🎯 Provider-Specific Examples #

Deep dive into specific providers

• OpenAI - GPT models, DALL-E, reasoning, assistants
• Anthropic - Claude models, extended thinking, file handling
• Google - Gemini models and multi-modal capabilities
• DeepSeek - Cost-effective reasoning models
• Groq - Ultra-fast inference
• Ollama - Local models and privacy-focused AI
• ElevenLabs - Advanced voice synthesis and recognition
• Others - XAI Grok and emerging providers

🎪 Real-World Use Cases #

Complete application examples

• chatbot.dart - Complete chatbot with personality and context management
• cli_tool.dart - Command-line AI assistant with multiple providers
• web_service.dart - HTTP API with AI capabilities, authentication, and rate limiting

🌟 Real-World Application #

Actively developed application built with LLM Dart

• Yumcha - Cross-platform AI chat application actively developed by the creator of LLM Dart, showcasing real-world integration with multiple providers, real-time streaming, and advanced features

🔗 MCP Integration #

Connect LLMs with external tools

• mcp_concept_demo.dart - 🎯 START HERE - Core MCP concepts
• simple_mcp_demo.dart - Working MCP + LLM integration
• test_all_examples.dart - 🧪 ONE-CLICK TEST - Test all examples
• Advanced MCP examples - Custom servers, tool bridges, and more

📖 Complete Examples Guide - Organized learning paths, detailed documentation, and best practices.

Contributing #

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

License #

This project is licensed under the MIT License - see the LICENSE file for details.

This library is inspired by the Rust graniet/llm library and follows similar patterns adapted for Dart.