flutter_local_llm 1.0.0 
flutter_local_llm: ^1.0.0 copied to clipboard
Published 25 days ago • 
therohitsoni.in
therohitsoni.inHigh-performance, production-grade Flutter plugin for on-device local LLM inference using llama.cpp via Dart FFI and Native Assets.
1.0.0 #
- Production Release of
flutter_local_llm:- Stable high-performance on-device inference using
llama.cppfor Flutter. - Hardware Acceleration:
- Apple Silicon GPU acceleration via Metal (
GGML_USE_METAL) for iOS and macOS. - OpenMP and Vulkan acceleration options for Android and desktop platforms.
- Multi-threaded AVX2 acceleration for x86_64 desktop platforms.
- Apple Silicon GPU acceleration via Metal (
- Dart FFI & Native Assets:
- Direct non-blocking token streaming using
NativeCallable<TokenCallbackNative>.listener. - Dart Native Assets (
hooks/build.dart) with dual CMake build pipeline.
- Direct non-blocking token streaming using
- Conversational AI & Session Management:
- Multi-turn chat context management with automatic KV-cache sliding window truncation.
- Pre-built prompt formatters for ChatML, Llama-3, Gemma, Mistral, and custom templates.
- Structured Outputs:
- GBNF (GGML BNF) grammar engine for schema-constrained JSON outputs.
- Utilities & UI:
- Resumable
ModelDownloaderwith HTTP Range requests, download speed/ETA telemetry, and SHA-256 verification. - Local LLM Studio example application with interactive Model Hub and real-time performance telemetry HUD.
- Resumable
- Stable high-performance on-device inference using
0.1.0 #
- Initial release preview.
Metadata
1
likes
160
points
148
downloads
Documentation
Publisher
Weekly Downloads
Metadata
High-performance, production-grade Flutter plugin for on-device local LLM inference using llama.cpp via Dart FFI and Native Assets.
Repository (GitHub)
View/report issues
Topics
#llm #llama-cpp #on-device #ai #ffi
License
MIT (license)
Dependencies
crypto, ffi, flutter, http, meta, path, path_provider
More
Packages that depend on flutter_local_llm
Packages that implement flutter_local_llm