Lora SDK #

LoraSDK is a Flutter package that enables easy integration of local Large Language Models (LLM) into your Flutter applications. It provides a streamlined interface for managing and running LLM models locally on device.

Features #

• Local LLM model execution
• Streaming text generation
• Model download and warmup progress monitoring
• Built-in state management
• Performance metrics collection
• Error handling

Installation #

run the following commands:

flutter pub add lora_sdk

And add these dependencies to your pubspec.yaml:

dependencies:
  lora_sdk: ^1.0.4
  fllama:
    git:
      url: https://github.com/Telosnex/fllama.git
      ref: main

Getting Started #

Complete Example #

For a quick start with our Lora SDK, you can test out a simple example application. Clone the test application repository using the following command:

git clone "https://github.com/peekaboolabs-appdev/lora-sdk-testapp.git"

This repository contains a basic implementation that demonstrates the core functionality of the Lora SDK. It serves as a practical reference for understanding how to integrate and utilize the SDK in your own applications.

After cloning the repository, you can run the test application on both iOS and Android devices using the following Flutter commands:

cd lora_sdk_test_app
flutter clean
flutter pub get
flutter run

You can test the application on both iOS and Android platforms to see the Lora SDK in action. Make sure you have set up your development environment with Flutter and the necessary platform-specific requirements.

Initialize SDK #

final myLlmProvider = FllamaWrapper();
final sdk = await LoraSdk.initialize(
  licenseKey: 'your-license-key',
  llmProvider: myLlmProvider,
);

Basic Usage #

// Download model if not already downloaded
await sdk.downloadModel(
  onProgress: (progress) => print('Download progress: ${progress * 100}%'),
  onError: (error) => print('Error: $error'),
);

// Warmup the model
await sdk.warmup(
  onProgress: (progress) => print('Warmup progress: ${progress * 100}%'),
  onError: (error) => print('Error: $error'),
);

// Generate text
final response = await sdk.generateResponse("Your input text");
print(response);

// Or use streaming response
sdk.generateStreamResponse("Your input text")
    .listen((chunk) => print(chunk));

State Management #

The SDK provides a LoraState class to track the current state:

enum ModelStatus {
  notInitialized, // SDK is not initialized
  initializing, // SDK is initializing
  initialized, // SDK initialization completed, model download required
  downloading, // Model is downloading
  downloaded, // Model download completed, warming up required
  warming, // Model is warming up
  ready, // Model is ready for use (warming up completed)
  error // Error state
}

class LoraState {
  final ModelStatus modelStatus; // Current model status
  final bool isInitialized; // Whether SDK initialization is completed
  final bool isWarmedUp; // Whether warming up is completed
  final bool isModelDownloaded; // Whether model download is completed
  final double? downloadProgress; // Download progress (0.0 ~ 1.0)
  final double? warmupProgress; // Warming-up progress (0.0 ~ 1.0)
  final String? errorMessage; // Error message
}

Monitor state changes using ValueListenableBuilder:

ValueListenableBuilder<LoraState>(
  valueListenable: sdk.loraState,
  builder: (context, state, _) {
    return Text('Current status: ${state.modelStatus}');
  },
);

License #

This SDK requires a license key to operate. You can obtain your license key by logging into our website:

1. Visit our website
2. Log in to your account (or create one if you haven't already)
3. Navigate to the dashboard
4. Find your license key in the "License Key" section

Each license key is tied to your account and has specific usage limitations. Please refer to our website for detailed licensing terms and conditions.

Requirements #

• Flutter SDK: >=3.0.0
• Dart SDK: >=3.0.0
• iOS: >=13.0
• Android: minSdk >=23
• Supported platforms: iOS, Android

Additional Information #

For bug reports and feature requests, please file an issue on our GitHub repository at this link.