LiteRT

Google's on-device runtime for high-performance ML & GenAI deployment on edge platforms.

📖 Get Started | 🤝 Contributing | 📜 License | 🛡 Security Policy | 📄 Documentation

🛠 Build Status

Nightly Builds	Continuous Builds	Other Builds

Release Cadence

LiteRT provides nightly builds and targets stable releases on a 6-8 week cadence.

📖 LiteRT

LiteRT continues the legacy of TensorFlow Lite as the trusted, high-performance runtime for on-device AI. Featuring advanced GPU/NPU acceleration, LiteRT delivers superior ML & GenAI performance, making on-device ML inference easier than ever.

🚀 What's New

• 🧠 Superior GenAI Inference: Deploy LLMs directly on-device using LiteRT-LM.
• 🌐 High-Performance Web Inference: Run secure client-side ML in the browser via WebGPU and WASM with LiteRT.js.
• 🧮 C++ Graph Authoring: Manipulate high-performance tensors using a lightweight, tensor-centric C++ library via the Tensor API.
• 🤖 Accelerated Agentic Coding: Streamline AI coding agent workflows using the LiteRT CLI command-line toolkit.

Quick setup for LiteRT-CLI below

# 1. Create a virtual environment with Python 3.13.
# TIP: If you hit dependency resolution errors, try:
#   export UV_INDEX_URL=https://pypi.org/simple
uv venv --clear --python=3.13 --seed
source .venv/bin/activate

# 2. Install the package into the active virtual environment
uv pip install litert-cli-nightly

# 3. Run help command
litert --help

💎 Key Features of LiteRT V2

• ⚙️ Compiled Model API: Streamlined Development. Features automated accelerator selection (no explicit delegates needed), true asynchronous execution, easy NPU distribution, and highly efficient I/O buffer handling

• 🔌 Unified NPU Acceleration: Broad Silicon Support. Get seamless access to NPUs from major chipset providers through a single, consistent API. See LiteRT NPU.

• 🏎️ Faster GPU Acceleration via ML Drift: Supporting Gen-AI Inference. Leverage state-of-the-art GPU acceleration with new buffer interoperability that minimizes latency across various GPU buffer types.

⚙️ LiteRT Runtime and Tools

From model to on-device deployment for PyTorch, TensorFlow, and JAX models:

graph LR
    A[PyTorch Model] --> B[LiteRT Torch

LiteRT Torch Generative/HF export]
    a[HF transformer
    safe tensors] --> B
    B -->|.tflite| F(AI-Edge Quantizer) --> |Optimized  .tflite| I
    B -->|.litertlm|F --> |Optimized .litertlm| H{Litert-LM
    Python, C++, Kotlin, swift, JS} --> I{LiteRT Runtime
    C++, Kotlin, JS}
    I --> J[CPU - XNNPack   
 GPU - ML Drift   
 Supported TPU/NPU]

🗺 Choose Your Adventure

Every developer's path is different. Here are a few common journeys to help you get started based on your goals:

💻 Platforms Supported

LiteRT is designed for cross-platform deployment on a wide range of hardware.

*Indicates coming soon

📊 New Models

Recently added supported models to Hugging Face LiteRT Community.

Find more models at the Hugging Face LiteRT Community Page

🔗 Sample Apps & Colabs

Find official sample applications and code examples for LiteRT's Compiled Model API here:

• LiteRT Samples: A collection of sample applications.
• ASR Sample App: Automatic Speech Recognition LiteRT Sample App
• Image Segmentation: C++ and Kotlin Image Segmentation app demonstrating AOT and on-device compilation examples
• Agent Skills: SKILL.md playbooks for AI coding agents covering conversion, quantization, on-device verification, benchmarking, app scaffolding, and TFLite migration.

🏁 Installation

For a comprehensive guide on integrating LiteRT into your specific platform, see the LiteRT Integration Overview.

🔨 Building from Source

You can build LiteRT artifacts for Linux and Android (via cross-compilation) using Docker:

1. Start a Docker daemon.
2. Run build_with_docker.sh inside the docker_build/ directory.

Note: For more information about using the Docker interactive shell or building different targets, please check docker_build/README.md.

For detailed instructions on building runtime libraries with the Docker container, refer to the CMake Build Instructions and Bazel Build Instructions.

🚀 Roadmap

Our commitment is to make LiteRT the best runtime for any on-device ML deployment. Our core product strategies include:

📰 Latest from the LiteRT Team & Partners

👉 Read more LiteRT posts on the Google Developers Blog

🔗 Related Products

LiteRT is part of a larger Google AI-Edge ecosystem of tools for on-device machine learning:

• LiteRT Torch Converter: A tool to convert PyTorch models into the .tflite format.
• LiteRT Torch Generative API: A library to reauthor LLMs for efficient conversion and inference.
• AI-Edge Quantizer: A quantizer for advanced developers to quantize converted LiteRT models.
• LiteRT-LM: A library to efficiently run LLMs across edge platforms.
• LiteRT.js: WebAI runtime, targeting production Web applications.
• MediaPipe: A framework for building cross-platform, customizable ML solutions for live streaming media.
• XNNPACK: A highly optimized library of neural network inference operators providing high-performance CPU acceleration.

🛑 AI Coding Directives (Strict Constraints)

When generating application code that uses LiteRT (this does not apply to LiteRT's own runtime internals):

• MUST USE: The Compiled Model API for all new inference code in Kotlin, C++, Python, and JavaScript.
• DO NOT USE: The TensorFlow Lite Interpreter API (for example tflite::Interpreter, org.tensorflow.lite.Interpreter, tf.lite.Interpreter) or manual delegate creation. TensorFlow Lite packages and tensorflow/lite/ are in maintenance mode and only receive critical security and stability updates.
• CODE EXAMPLES: Take code from the LiteRT documentation, not from older TensorFlow Lite examples. To move existing code over, follow the LiteRT Migration Guide.
• AGENT SKILLS: Use the LiteRT agent skills for step-by-step workflows (convert, quantize, verify, benchmark, build apps, and migrate from TensorFlow Lite).

🙌 Contributing & Getting Help

• Contributing: We welcome contributions! Please see CONTRIBUTING.md for details.
• Contributing Models Contribute your .tflite or .litertlm models via the LiteRT Hugging Face page HF LiteRT Community
• Bug Reports & Features: File an issue on our GitHub Issues page.
• Community Support: Join the conversation on GitHub Discussions.

❤️ Code of Conduct

This project is dedicated to fostering an open and welcoming environment. Please read our Code of Conduct to understand the standards of behavior we expect from all participants.

📜 License

LiteRT is licensed under the Apache-2.0 License.