alvinreal/awesome-opensource-ai — stars, momentum and ranking

Curated list of the best truly open-source AI projects, models, tools, and infrastructure. Daily updated.

Awesome Open Source AI

Curated open-source artificial intelligence models, libraries, infrastructure, and developer tools.

1. Core Frameworks & Libraries
2. Model Codebases & Model Families
3. Inference Engines & Serving
4. Agentic AI & Multi-Agent Systems
5. Retrieval-Augmented Generation (RAG) & Knowledge
6. Generative Media Tools
7. Training & Fine-tuning Ecosystem
8. MLOps / LLMOps & Production
9. Evaluation, Benchmarks & Datasets
10. AI Safety, Alignment & Interpretability
11. Specialized Domains
12. User Interfaces & Self-hosted Platforms
13. Developer Tools & Integrations
14. Resources & Learning

About this list

Awesome Open Source AI is a curated list of open-source projects for people building with AI.

The goal is to help readers find useful models, libraries, tools, infrastructure, datasets, and learning resources without sorting through a directory dump.

Projects do not need a minimum number of GitHub stars to be included. Stars can be useful context, but they are only one signal. A smaller project may belong here if it is useful, well-maintained, technically interesting, clearly documented, or important to a specific part of the AI ecosystem.

Good entries should have a clear reason to exist. They should help people build, study, run, evaluate, or understand AI systems.

1. Core Frameworks & Libraries

Core libraries and frameworks used to build, train, and run AI and machine learning systems.

Deep Learning Frameworks

PyTorch - Dynamic computation graphs, Pythonic API, dominant in research and production. The current standard for most frontier AI work.
TensorFlow - End-to-end platform with excellent production deployment, TPU support, and large-scale serving tools.
JAX - High-performance numerical computing with composable transformations (JIT, vmap, grad). Rising favorite for research and scientific ML.
Flax - Neural network library for JAX, designed for flexibility. Apache-2.0 licensed.
dm-haiku - JAX-based neural network library from Google DeepMind. Elegant functional API with state management, widely used in DeepMind's research. Apache 2.0 licensed.
Equinox - Elegant easy-to-use neural networks and scientific computing in JAX. Callable PyTrees with filtered transformations, seamless interoperability with the JAX ecosystem. Apache 2.0 licensed.
Diffrax - Numerical differential equation solvers in JAX. Autodifferentiable and GPU-capable ODE/SDE/CDE solvers for scientific machine learning and neural differential equations. Apache 2.0 licensed.
vit-pytorch - Comprehensive Vision Transformer (ViT) implementations in PyTorch. Reference implementations of all major vision transformer variants including ViT, DeiT, Swin, and more. MIT licensed.
NumPyro - Probabilistic programming with NumPy powered by JAX for autograd and JIT compilation. Bayesian modeling and inference at scale.
Keras - High-level, beginner-friendly API that now runs on multiple backends (TensorFlow, JAX, PyTorch). Perfect for rapid experimentation.
tinygrad - Minimalist deep learning framework with tiny code footprint. The "you like PyTorch? you like micrograd? you love tinygrad!" philosophy - simple yet powerful.
PaddlePaddle - Industrial deep learning platform from Baidu serving 23+ million developers and 760,000+ companies. China's first independent R&D framework with advanced distributed training and deployment capabilities.
PyTorch Geometric - Library for deep learning on irregular input data such as graphs, point clouds, and manifolds. Part of the PyTorch ecosystem.
timm (PyTorch Image Models) - The largest collection of PyTorch image encoders and backbones. 900+ pretrained models including ResNet, EfficientNet, Vision Transformer, ConvNeXt, and more with training and inference scripts. Apache 2.0 licensed.
Triton - Language and compiler for writing highly efficient custom deep-learning primitives. Powers kernel optimizations in PyTorch, JAX, and other frameworks. MIT licensed.
GGML - Tensor library for machine learning. The foundational C/C++ library powering llama.cpp and many on-device inference engines. MIT licensed.
MLX - Array framework for machine learning on Apple silicon. Efficient unified memory design with NumPy-like API, automatic differentiation, and multi-device support. MIT licensed.
notorch - Neural network framework in pure C with reverse-mode automatic differentiation, model training, and GGUF inference, without a Python runtime.

High-Performance Compute Libraries

oneDNN - oneAPI Deep Neural Network Library. Cross-platform performance library of basic building blocks for deep learning, optimized for Intel CPUs, GPUs, and Arm architectures. Apache 2.0 licensed.
ONNX - Open standard for machine learning interoperability. Open Neural Network Exchange provides an open ecosystem that empowers AI developers to choose the right tools as their project evolves. Apache 2.0 licensed.
IREE - Retargetable MLIR-based machine learning compiler and runtime toolkit. Lowers ML models to unified IR that scales from datacenter to mobile and edge deployments. Apache 2.0 licensed.
Modular Platform - Open-source AI compute and programming platform built around the MAX Engine and Mojo programming language.

Rust ML Frameworks

Burn - Next-generation deep learning framework in Rust. Backend-agnostic with CPU, GPU, WebAssembly support.
Candle (Hugging Face) - Minimalist ML framework for Rust. PyTorch-like API with focus on performance and simplicity.
linfa - Comprehensive Rust ML toolkit with classical algorithms. scikit-learn equivalent for Rust with clustering, regression, and preprocessing.

Julia ML Frameworks

Flux.jl - 100% pure-Julia ML stack with lightweight abstractions on top of native GPU and AD support. Elegant, hackable, and fully integrated with Julia's scientific computing ecosystem.
MLJ.jl - Comprehensive Julia machine learning framework providing a unified interface to 200+ models with meta-algorithms for selection, tuning, and evaluation. MIT licensed.
ModelingToolkit.jl - High-performance symbolic-numeric modeling framework for scientific machine learning. Automatically generates fast functions for model components like Jacobians and Hessians with automatic sparsification and parallelization. MIT licensed.

NLP & Transformers

spaCy (Explosion AI) - Industrial-strength natural language processing with 75+ languages, transformer pipelines, and production-grade NER, parsing, and text classification.
Transformers (Hugging Face) - The de facto standard library for pretrained NLP models. 1M+ models, 250,000+ downloads/day. BERT, GPT, Llama, Qwen, and hundreds more.
sentence-transformers - Classic library for sentence and image embeddings.
tokenizers (Hugging Face) - Fast state-of-the-art tokenizers for training and inference.
fairseq2 - FAIR Sequence Modeling Toolkit 2. Complete rewrite of fairseq with modern PyTorch APIs, native support for LLM training (70B+ models), vLLM integration, and first-party recipes for instruction finetuning and preference optimization. MIT licensed.
LibreTranslate - Self-hosted machine translation API powered by the Argos Translate engine. AGPL-3.0 licensed.

Data Processing & Manipulation

Pandas - The gold standard for data analysis and manipulation in Python.
Polars - Blazing-fast DataFrame library (Rust backend) - modern alternative to Pandas for large-scale workloads.
cuDF - GPU DataFrame library from RAPIDS. Accelerates Pandas workflows on NVIDIA GPUs with zero code changes using cuDF.pandas accelerator mode.
Dask - Parallel computing for big data - scales Pandas/NumPy/scikit-learn to clusters.
DataFlow - LLM-ready data preparation system for turning raw PDFs, conversations, code, databases, and other sources into SFT, QA, and RAG-ready datasets.
NumPy - Fundamental array computing library that powers almost every AI stack.
SciPy - Scientific computing algorithms (optimization, linear algebra, statistics, signal processing).
CuPy - NumPy and SciPy-compatible array library for GPU-accelerated computing in Python.
NetworkX - Creation, manipulation, and study of complex networks. The foundational graph analysis library for Python data science.
cuGraph - GPU graph analytics library with NetworkX-compatible API. 10-100x faster than CPU for large-scale graph algorithms. Apache 2.0 licensed.
Datashader - High-performance large data visualization. Renders billions of points interactively without aggregation artifacts. BSD-3-Clause licensed.
Zarr - Chunked, compressed, N-dimensional array storage. Scalable tensor data format optimized for cloud and parallel computing. MIT licensed.
NVIDIA DALI - GPU-accelerated data loading and augmentation library with highly optimized building blocks for deep learning applications. Apache 2.0 licensed.
Narwhals - Lightweight compatibility layer between DataFrame libraries. Write Polars-like code that works seamlessly across Pandas, Polars, cuDF, Modin, and more. MIT licensed.
Ibis - Portable Python dataframe library with 20+ backends. Write Pandas-like code that runs locally with DuckDB or scales to production databases (BigQuery, Snowflake, PostgreSQL) by changing one line. Apache 2.0 licensed.
skrub - Machine learning with dataframes for dirty categorical data. Preprocessing and feature engineering for heterogeneous data with seamless Pandas/Polars integration. BSD-3-Clause licensed.
Oxen - Lightning fast data version control for machine learning. Optimized for large datasets with efficient diffing, branching, and collaboration. Apache 2.0 licensed.
Pandera - Statistical data testing and validation for dataframes. Pydantic-like API for Pandas, Polars, and other dataframe libraries with type hints and lazy validation. MIT licensed.
Snorkel - System for quickly generating training data with weak supervision. Programmatically label, build, and manage training data using labeling functions and probabilistic consensus models. Powers Snorkel Flow and used by Google, Apple, and Intel. Apache 2.0 licensed.
DuckDB - High-performance analytical in-process SQL database system. Fast, reliable, portable, and easy to use with rich SQL dialect support. Perfect for data processing and analytics workloads. MIT licensed.
FiftyOne - Visual AI development toolkit for visualizing, labeling, and evaluating visual datasets and models. Supercharges computer vision workflows with dataset exploration and model analysis. Apache 2.0 licensed.
Label Studio - Multi-type data labeling and annotation tool with standardized output format. Configurable interface for images, text, audio, video, and time series with ML-assisted labeling. Apache 2.0 licensed.
Delta Lake - Open-source storage framework enabling Lakehouse architecture with ACID transactions, scalable metadata handling, and unified batch/streaming processing. Apache 2.0 licensed.
Apache Iceberg - High-performance open table format for huge analytic tables. Brings SQL table reliability to big data with time travel, hidden partitioning, and schema evolution. Works with Spark, Trino, Flink, Presto, Hive and Impala. Apache 2.0 licensed.
Apache Hudi - Open data lakehouse platform for ingesting, indexing, storing, serving, transforming and managing data across cloud environments. Supports upserts, deletes and incremental processing on big data with built-in ingestion tools for Spark and Flink. Apache 2.0 licensed.
lakeFS - Data version control for your data lake that transforms object storage into Git-like repositories. Enables atomic, versioned data lake operations with branching, committing, and merging for data pipelines. Apache 2.0 licensed.
Apache Airflow - Platform to programmatically author, schedule, and monitor workflows. Industry-standard orchestration for data pipelines and ML workflows with 500+ integrations. Apache 2.0 licensed.
Apache Spark - Unified analytics engine for large-scale data processing. In-memory cluster computing with high-level APIs in Python, Scala, Java, and R. Powers MLlib for distributed machine learning and Structured Streaming for real-time data. Apache 2.0 licensed.
Apache Flink - Stream processing framework with powerful batch and streaming capabilities. High-throughput, low-latency runtime with exactly-once processing guarantees. Ideal for real-time AI inference pipelines and event-driven ML applications. Apache 2.0 licensed.
Apache Beam - Unified programming model for batch and streaming data processing. Write pipelines once, run anywhere on Flink, Spark, or Google Cloud Dataflow. Portable, extensible, and enterprise-ready for AI data pipelines. Apache 2.0 licensed.
Scrapy - Fast, high-level web crawling and scraping framework for Python. Extract structured data from websites at scale with built-in support for handling common challenges like pagination, cookies, and concurrent requests. BSD-3-Clause licensed.
Temporal - Durable execution platform for reliable workflow orchestration. Build resilient data pipelines and ML workflows that survive failures and continue execution exactly where they left off. MIT licensed.
Luigi - Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
Mage.ai - Modern open-source data pipeline tool for integrating and transforming data. AI-native ETL/ELT platform with 100+ integrations, real-time monitoring, and collaborative features. Apache 2.0 licensed.
Hamilton - Declarative dataflow framework for building testable, modular, self-documenting data pipelines. Encode lineage and metadata directly in Python functions. Originally from Stitch Fix, now Apache incubating. Apache 2.0 licensed.
D-Tale - Visualizer for Pandas data structures with a Flask back-end and React front-end. Interactive data exploration with charting, filtering, and code export. LGPL-2.1 licensed.
Sweetviz - Beautiful, high-density visualizations for exploratory data analysis in two lines of code. Self-contained HTML reports for dataset comparison and target analysis. MIT licensed.
TextAttack - Python framework for adversarial attacks, data augmentation, and model training in NLP. Augment datasets to increase model robustness and generate adversarial examples. MIT licensed.
uv - An extremely fast Python package and project manager, written in Rust. 10-100x faster than pip with built-in virtual environment management, dependency resolution, and lockfiles. Essential for modern AI/ML development workflows. Apache 2.0 and MIT dual-licensed.
Vector - A high-performance observability data pipeline for collecting, transforming, and routing logs and metrics. Real-time data processing with 50+ sources and sinks including Kafka, S3, and Elasticsearch. Ideal for AI/ML log processing and data ingestion. MPL 2.0 licensed.

Classical ML & Gradient Boosting

scikit-learn - Industry-standard library for traditional machine learning (classification, regression, clustering, pipelines).
XGBoost - Scalable, high-performance gradient boosting library. Still dominates Kaggle and tabular competitions.
LightGBM - Microsoft's ultra-fast gradient boosting framework, optimized for speed and memory.
CatBoost - Gradient boosting that handles categorical features natively with great out-of-the-box performance.
sktime - Unified framework for machine learning with time series. scikit-learn compatible API for forecasting, classification, clustering, and anomaly detection.
StatsForecast - Lightning-fast statistical forecasting with ARIMA, ETS, CES, and Theta models. Optimized for high-performance time series workloads.
MLForecast - Scalable machine learning for time series forecasting. Train any sklearn-compatible model on millions of time series with efficient feature engineering. Apache 2.0 licensed.
cuML - GPU-accelerated machine learning algorithms with scikit-learn compatible API. 10-50x faster than CPU implementations for large datasets. Apache 2.0 licensed.
SynapseML - Distributed machine learning on Apache Spark. Scalable, composable APIs for text analytics, vision, anomaly detection with seamless Python/Scala/R/.NET integration. MIT licensed.
Darts - User-friendly forecasting and anomaly detection for time series. Unifies classical statistical models (ARIMA, ETS) with modern neural networks (N-BEATS, TFT, DeepAR) in a single scikit-learn compatible API. Apache 2.0 licensed.
PyTorch Forecasting - Time series forecasting with PyTorch. Multiple neural architectures (N-BEATS, TFT, DeepAR) with in-built interpretation capabilities, built on PyTorch Lightning for distributed training. MIT licensed.

Data Engineering & Feature Stores

DataHub - The #1 open-source metadata platform for data and AI. Data discovery, governance, and observability with 80+ connectors, column-level lineage, and AI assistant integration. Originally built at LinkedIn. Apache 2.0 licensed.
OpenMetadata - Unified metadata platform for data discovery, observability, and governance. Column-level lineage, semantic search, and team collaboration with 70+ data service connectors. Apache 2.0 licensed.
Amundsen - Data discovery and metadata engine from Lyft. PageRank-style search for data resources with usage-based ranking. LF AI & Data Foundation project. Apache 2.0 licensed.

Data Transformation & Analytics Engineering

Apache Ossie - Vendor-neutral specification to standardize semantic models across analytics, BI, and AI agent platforms. Apache 2.0 licensed.
dbt-core - Transform data using software engineering best practices. The industry-standard framework for analytics engineering with 15M+ monthly downloads. Enables version control, testing, and documentation for SQL transformations. Apache 2.0 licensed.
SQLMesh - Scalable and efficient data transformation framework with dbt compatibility. Features automatic data lineage, time travel, and virtual data environments for testing. Optimized for large-scale data warehouses. Apache 2.0 licensed.
SLayer - Semantic layer for AI-powered data analytics. Allows AI agents to describe data models and query the data using an expressive format with measures, dimensions, and filters, without writing raw SQL. MCP, CLI, API, and Python clients. Embeddable as a Python library. MIT licensed.
WrenAI - Open-source Generative BI engine and context layer for AI agents to query databases and produce trusted SQL and dashboards. Apache 2.0 licensed.

Data Quality & Validation

Deequ - Library built on top of Apache Spark for defining "unit tests for data". Measures data quality in large datasets with constraint verification, anomaly detection, and incremental validation. Used at Amazon for production data quality. Apache 2.0 licensed.
Great Expectations - Always know what to expect from your data. Data validation, profiling, and documentation for data pipelines. Apache 2.0 licensed.
ydata-profiling - One line of code for comprehensive data quality profiling and exploratory data analysis. Generates detailed reports for Pandas and Spark DataFrames including statistics, correlations, missing values, and data quality alerts. MIT licensed.
Soda Core - Data contracts engine for the modern data stack. Define data quality checks in YAML and automatically validate schema and data across your pipelines. Supports 20+ data sources including Snowflake, BigQuery, and PostgreSQL. Apache 2.0 licensed.
TFX (TensorFlow Extended) - End-to-end platform for deploying production ML pipelines. Data validation, transformation, model training, and serving with TensorFlow. Powers Google's production ML infrastructure. Apache 2.0 licensed.

Data Labeling & Annotation

Doccano - Open-source text annotation tool for machine learning practitioners. Features text classification, sequence labeling, and sequence-to-sequence tasks for sentiment analysis, NER, and summarization. MIT licensed.
OpenRefine - Free, open-source power tool for working with messy data. Clean, transform, and extend data with web services. Formerly Google Refine. BSD-3-Clause licensed.

AutoML & Hyperparameter Optimization

Optuna - Modern, define-by-run hyperparameter optimization with pruning and visualizations. Extremely popular in 2026.
AutoGluon - AWS AutoML toolkit for tabular, image, text, and multimodal data - state-of-the-art with almost zero code.
FLAML - Microsoft's fast & lightweight AutoML focused on efficiency and low compute.
Katib (Kubeflow) - Kubernetes-native AutoML for hyperparameter tuning, early stopping, and neural architecture search. Framework-agnostic with support for TensorFlow, PyTorch, XGBoost, and custom training operators. Apache 2.0 licensed.

Interactive ML Apps & Notebooks

Streamlit - The fastest way to build and share data apps. Transform Python scripts into beautiful web applications with minimal code. Widely used for ML model demos, data visualization, and internal tools.
Gradio - Build and share delightful machine learning apps, all in Python. The de facto standard for creating interactive ML demos with automatic UI generation from function signatures. Powers thousands of Hugging Face Spaces.
Marimo - A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.

Model Training & Optimization Utilities

Hugging Face Accelerate - Simple API to make training scripts run on any hardware (multi-GPU, TPU, mixed precision) with minimal code changes.
DeepSpeed - Microsoft's deep learning optimization library for extreme-scale training (ZeRO, offloading, MoE).
FlashAttention - Fast exact attention kernels that reduce memory usage and accelerate transformer training and inference.
xFormers - Optimized transformer building blocks and attention operators for PyTorch.
PyTorch Lightning - High-level wrapper for PyTorch that removes boilerplate and adds best practices.
fastai - Deep learning library providing practitioners with high-level components for state-of-the-art results. Built on PyTorch with a focus on usability and transfer learning. Apache 2.0 licensed.
PyTorch Ignite - High-level library for training and evaluating neural networks in PyTorch with an engine, events & handlers system for maximum flexibility. BSD-3-Clause licensed.
ONNX Runtime - High-performance inference and training for ONNX models across hardware.
einops - Flexible, powerful tensor operations for readable and reliable code. Supports PyTorch, JAX, TensorFlow, NumPy, MLX.
safetensors - Simple, safe way to store and distribute tensors. Fast, secure alternative to pickle for model serialization.
torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications. 80+ metrics with built-in distributed synchronization.
torchao - PyTorch native quantization and sparsity for training and inference. Drop-in optimizations for production deployment.
SHAP - Game theoretic approach to explain the output of any machine learning model. Industry standard for model interpretability.
skorch - scikit-learn compatible neural network library that wraps PyTorch. Seamlessly integrate PyTorch models with scikit-learn pipelines, grid search, and cross-validation.
Composer - Supercharge your model training. MosaicML's PyTorch training library with built-in algorithms for efficient training (FSDP, gradient compression, progressive resizing) and seamless distributed training on large-scale clusters. Apache 2.0 licensed.
NVIDIA Apex - PyTorch extension for mixed precision training and distributed training optimizations. Powers many production deep learning workloads with tools for automatic mixed precision (AMP), distributed data parallel, and fused optimizers. BSD-3-Clause licensed.

2. Model Codebases & Model Families

Canonical model-family repositories with useful code, recipes, evaluation tools, or engineering context. This is not a complete model leaderboard; use Hugging Face and model hubs for up-to-date weight discovery.

Language Model Families

RWKV - Attention-free language model architecture with linear-time inference, training code, inference examples, and an active open-source ecosystem.
MiniCPM - Compact open model family with practical code, deployment notes, and active edge/on-device focus.
GPT-OSS - OpenAI open-weight model repository with inference examples, recipes, and deployment guidance.
Mamba - State Space Model implementation with pretrained checkpoints, architecture code, and research tooling for efficient long-sequence modeling.
GPT-NeoX - Large-scale language model training codebase from EleutherAI with distributed training support and historical open-model importance.
GLM-5 - Open-source mixture-of-experts language model family optimized for long-horizon planning, agentic tasks, and coding. Apache 2.0 licensed.
Bonsai Demo - Local runtime and model package for the Bonsai ternary reasoning model family, with GGUF and MLX support for on-device text, vision, and tool-calling workloads. Apache-2.0 licensed.

Multimodal & Vision-Language Codebases

OpenCLIP - Open implementation of CLIP with training code, pretrained models, and zero-shot evaluation tooling.
OmniParser - Vision-based GUI parsing model and tooling for computer-use agents.
MiniCPM-V - Compact vision-language model family with edge-focused deployment examples and strong OCR-oriented use cases.
Eagle - NVIDIA multimodal model codebase with open checkpoints and reusable research materials for vision-language and video-language tasks.
Moondream - Small vision-language model with practical inference examples for edge and real-time image understanding.
NVIDIA Cosmos - Open platform of world models, tokenizers, and post-training tools designed for physical AI, robotics, and autonomous systems.

Speech & Audio Model Codebases

Whisper - Canonical open speech-to-text model codebase with widespread ecosystem support and many downstream implementations.
FunASR - Speech recognition toolkit with pretrained models, streaming support, diarization, VAD, and production-oriented examples.
NVIDIA NeMo - Scalable framework and model codebase for speech, language, and multimodal AI with recipes and deployment guidance.
Sherpa-ONNX - Complete speech toolkit with ASR, TTS, diarization, source separation, and VAD across embedded and edge environments via ONNX Runtime.
MOSS-TTS - Open speech and sound generation family focused on expressive, long-form text-to-speech with streaming and multi-speaker support.
VoxCPM - Open-sourced tokenizer-free multilingual speech synthesis model with high-quality TTS and style transfer workflows.
VibeVoice - Open Frontier Voice AI toolkit spanning speech understanding, generation, and multilingual TTS workflows, with active research and deployment tooling.
SpeechBrain - PyTorch speech toolkit with recipes for ASR, TTS, speaker recognition, and speech enhancement.
Pocket TTS - Lightweight text-to-speech engine optimized for CPU inference with low latency and streaming support. MIT licensed.
transcribe.cpp - C/C++ speech-to-text inference library running 16+ model families on the ggml runtime with GPU acceleration.
Moonshine - Open-source on-device voice AI toolkit for low-latency speech-to-text, intent recognition, and text-to-speech.

3. Inference Engines & Serving

Inference runtimes, serving systems, and optimization tools for running models locally or in production.

Local / On-device Inference

llama.cpp - Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.
Ollama - Dead-simple local LLM runner with a one-line install, model registry, and OpenAI-compatible API.
Foundry Local - Open-source on-device AI platform covering discovery, model running, sandboxed execution, and evaluation of open models.
Potato OS - Linux distribution for fully local AI inference on Raspberry Pi 5 and 4, optimized for running open models at the edge.
MLC-LLM - Deployment engine that compiles and runs LLMs across browsers, mobile devices, and local hardware.
WebLLM - High-performance in-browser LLM inference engine. Runs models directly in the browser with WebGPU acceleration.
llama-cpp-python - Official Python bindings for llama.cpp.
KoboldCpp - User-friendly llama.cpp fork focused on role-playing and creative writing.
RamaLama - Container-centric tool for simplifying local AI model serving. Automatically detects GPUs, pulls optimized container images, and runs models securely in rootless containers with enterprise-grade isolation.
LiteRT - Google's production-ready on-device ML and GenAI deployment framework. Supports Android, iOS, Web, Desktop, and IoT targets with GPU/NPU acceleration via a unified edge-first runtime. Apache 2.0 licensed.
LiteRT-LM - Production-ready runtime for deploying LLMs on edge devices with low-latency inference and optimized hardware paths for mobile and embedded platforms.
exo - Run frontier AI locally by connecting all your devices into an AI cluster. Features automatic device discovery, RDMA over Thunderbolt for 99% latency reduction, topology-aware auto parallel, and tensor parallelism. Uses MLX backend for distributed inference across Apple Silicon devices. Apache 2.0 licensed.
ds4 - Native inference engine optimized for DeepSeek V4 and GLM models with Metal, CUDA, and ROCm support.
qwen3.8-flash-next-in-c - Native C implementation of Qwen3.8-Flash-Next for local CPU inference, with terminal chat, a resident OpenAI-compatible API, and reproducible benchmarks.
omlx - Apple-centric inference server for local-first AI workflows with model management, GPU orchestration, and OpenAI-compatible APIs for self-hosted deployment. Apache 2.0 licensed.
llmfit - Terminal tool and TUI that right-sizes LLM models to hardware specs and scores local compatibility across GPU, CPU, and RAM. MIT licensed.
Needle - Compact 45M-parameter foundation model and 14MB inference engine for tool calling and structured extraction on tiny devices. MIT licensed.
Colibri - Zero-dependency C inference engine that runs large Mixture-of-Experts models locally by streaming experts across disk, RAM, and VRAM.
Claude Code Local - MLX-native server that speaks the Anthropic Messages API so the unmodified Claude Code CLI runs against local models on Apple Silicon, with parsing for local models' tool-call formats. MIT licensed.
nemotron-omni-mlx - Pure MLX runtime for the vision and audio towers of NVIDIA Nemotron 3 Nano Omni on Apple Silicon, tested for parity against NVIDIA's PyTorch reference. MIT licensed.
Magnitude - Hardware-aware local inference engine that profiles host hardware, recommends suitable open models, and tunes execution across Apple Silicon, NVIDIA, AMD, and CPU. Apache 2.0 licensed.
llm-vram-calculator - Dependency-free TypeScript core that estimates inference memory (weights, KV cache, overhead) of any Hugging Face model from its config.json and safetensors metadata, covering MoE, MLA, sliding-window and linear-attention layers; runs in the browser at modelvram.com. MIT licensed.

High-performance Serving & API Servers

llm-d - Kubernetes-native distributed LLM inference framework. Donated to CNCF by RedHat, Google, and IBM. Intelligent scheduling, KV-cache optimization, and state-of-the-art performance across accelerators.
LMDeploy - Toolkit for compressing, deploying, and serving LLMs from OpenMMLab. 4-bit inference with 2.4x higher performance than FP16, distributed multi-model serving across machines.
vLLM - State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
vLLM-Omni - Multi-modal inference stack extending vLLM for image, audio, and video generation workloads with a unified serving interface. Apache 2.0 licensed.
LMCache - Supercharge LLM inference with the fastest KV Cache layer. 3-10x delay savings and GPU cycle reduction for multi-round QA and RAG. Integrates seamlessly with vLLM for distributed, high-throughput deployments. Apache 2.0 licensed.
vLLM Production Stack - Kubernetes-native production stack for vLLM inference. Automated deployment, autoscaling, and monitoring for enterprise-grade LLM serving. Built by the vLLM team for seamless integration.
Open Model Engine (OME) - Kubernetes operator for LLM serving with GPU scheduling and model lifecycle management across vLLM, SGLang, and TensorRT-LLM.
nano-vLLM - Minimalist vLLM implementation in ~1,200 lines of Python. Educational yet performant with prefix caching, tensor parallelism, and CUDA graph acceleration. Comparable inference speeds to full vLLM. MIT licensed.
SGLang - Next-gen serving framework with RadixAttention. Powers xAI's production workloads at 100K+ GPUs scale.
TensorRT-LLM - NVIDIA's official high-performance inference backend.
Aphrodite Engine - vLLM fork optimized for role-play and creative writing. Supports extensive quantization methods (AQLM, AWQ, GPTQ, GGUF, FP8) and modern samplers. Active development with multi-LoRA and speculative decoding support.
AIBrix - Cost-efficient and pluggable infrastructure components for GenAI inference. Kubernetes-native control plane for vLLM with distributed KV cache, heterogeneous GPU serving, and intelligent routing. Apache 2.0 licensed.
AISIX - Self-hosted Apache-2.0-licensed AI gateway written in Rust, with OpenAI-compatible endpoints, an Anthropic-compatible Messages endpoint, model routing, traffic policies, Prometheus metrics, and OTLP trace export.
Triton Inference Server - NVIDIA's production-grade open-source inference serving software. Supports multiple frameworks (TensorRT, PyTorch, ONNX) with optimized cloud and edge deployment.
mistral.rs - Fast, flexible Rust-native LLM inference engine built on Candle. Supports text, vision, audio, image generation, and embeddings with hardware-aware auto-tuning.
KTransformers - Flexible framework for heterogeneous CPU-GPU LLM inference and fine-tuning. Enables running large MoE models by offloading experts to CPU with BF16/FP8 precision support.
llamafile - Mozilla's single-file distributable LLM solution. Bundle model weights, inference engine, and runtime into one portable executable that runs on six OSes without installation.
Xinference - Unified, production-ready inference API for LLMs, speech, and multimodal models. Drop-in GPT replacement with single-line code changes. Supports thousands of models with auto-batching and distributed inference.
RTP-LLM (Alibaba) - Alibaba's high-performance LLM inference acceleration engine. Powers production LLM services across Taobao, Tmall, and Alibaba's international AI platform. Supports PagedAttention, FlashAttention, FlashDecoding, INT8/INT4 quantization, and heterogeneous hardware (GPU/ARM CPU/Intel). Apache 2.0 licensed.
LitServe (Lightning AI) - Minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling. 2x faster than FastAPI with built-in batching, streaming, and multi-GPU autoscaling. Apache 2.0 licensed.
LightLLM - Pure Python-based LLM inference and serving framework with lightweight design, easy extensibility, and high-speed performance. Integrates optimizations from FasterTransformer, TGI, vLLM, and SGLang.
TabbyAPI - FastAPI-based API server for ExLlamaV2/V3 backends. OpenAI-compatible API with support for model loading/unloading, embeddings, speculative decoding, multi-LoRA, and streaming.
GPUStack - GPU cluster manager that orchestrates inference engines like vLLM and SGLang. Automated engine selection, parameter optimization, and distributed multi-GPU deployment for high-performance AI workloads.
OpenLLM (BentoML) - Production-grade platform for running any open-source LLMs as OpenAI-compatible API endpoints. Supports 50+ models with built-in streaming, batching, and auto-acceleration. Apache 2.0 licensed.
Higress (Alibaba) - AI-native API gateway born from Alibaba's internal infrastructure with 2+ years of production validation. Provides unified LLM API and MCP (Model Context Protocol) management with enterprise-grade 99.99% availability. Apache 2.0 licensed.
NVIDIA Dynamo - Datacenter-scale distributed inference serving framework from NVIDIA. Orchestration layer above vLLM/SGLang/TensorRT-LLM with disaggregated serving, KV-aware routing, and automatic scaling. Built in Rust with Python extensibility. Apache 2.0 licensed.
Microsoft BitNet - Official inference framework for 1-bit LLMs (BitNet b1.58). Enables running large models on CPU with minimal memory footprint. Features custom kernels for ternary weight quantization and efficient matmul operations. MIT licensed.
FreeLLMAPI - OpenAI-compatible proxy gateway that stacks the free tiers of multiple LLM providers behind a single endpoint with automatic failover and rate tracking. MIT licensed.
OmniRoute - Unified AI gateway and proxy supporting over 230 providers with token compression, automatic failover, and routing strategies. MIT licensed.
Switchyard - Rust proxy and library for routing, protocol translation, and operational metrics across LLM backends and coding agents. Apache 2.0 licensed.
Bifrost - LLM gateway with a unified OpenAI-compatible API across providers, routing, load balancing, fallbacks, guardrails, and observability. Apache 2.0 licensed.
fitcheck - Zero-dependency Python CLI that plans LLM VRAM before you serve: real per-token KV-cache math, FITS/TIGHT/OOM verdicts, max safe context, and paste-ready vLLM/llama.cpp flags. MIT licensed.

Additional Inference Engines

DeepEP - Efficient expert-parallel communication library for large MoE models, improving throughput in distributed inference and training.
DeepGEMM - CUDA FP8/FMA GEMM kernels for efficient LLM inference and training at reduced precision.
AirLLM - Single-GPU 70B inference stack with strong memory/performance optimizations for local deployment on commodity hardware.
ThunderKittens - High-performance GPU kernel primitives for fast attention and matmul workflows used by LLM stacks.
Mirage Persistent Kernel - Compiler that fuses model execution into a single mega-kernel for tighter performance.
tt-metal - Operator and kernel toolkit for efficient LLM inference and low-level optimization on Tenstorrent hardware.
vLLM-Ascend - Hardware plugin for running vLLM on Huawei Ascend accelerators.
CTranslate2 - Fast inference engine for Transformer models supporting OpenNMT and Hugging Face models. Optimized for CPU and GPU with batching, quantization (INT8/FP16), and dynamic memory management. Powers faster-whisper and other production deployments. MIT licensed.
llama-swap - Intelligent model swapping proxy for llama.cpp. Enables seamless hot-swapping between different GGUF models without restarting the server, with automatic model loading/unloading and OpenAI-compatible API. MIT licensed.
optillm - Optimizing inference proxy for LLMs with load balancing, failover, and request routing across multiple providers and models. Improves reliability and performance for production deployments. Apache 2.0 licensed.
Fugusashi - Open-source intelligent model router with CMA-ES evolved routing weights, federated learning for privacy-preserving collaborative routing, and human-interpretable explanations for every decision. OpenAI-compatible API with web dashboard. MIT licensed.
OpenRoutiQ - Explainable, policy-controlled Python router for selecting models, providers, deployments, and reasoning levels, with optional outcome learning and an OpenAI-compatible proxy. MIT licensed.
mllm - Fast and lightweight multimodal LLM inference engine for mobile and edge devices. Optimized for running vision-language models on resource-constrained hardware with efficient memory management. MIT licensed.
shimmy - Python-free Rust inference server with OpenAI API compatibility. Supports GGUF and SafeTensors formats with hot model swap, auto-discovery, and single binary deployment for zero-dependency inference. Apache 2.0 licensed.
PowerInfer - High-speed LLM inference for local deployment on consumer GPUs. Achieves up to 11x speedup over llama.cpp on RTX 4090 by exploiting power-law neuron activation patterns. MIT licensed.
distributed-llama - Distributed LLM inference connecting home devices into a powerful cluster. More devices means faster inference via tensor parallelism over Ethernet. Supports Linux, macOS, Windows, ARM, and x86_64 AVX2 CPUs. MIT licensed.
ik_llama.cpp - High-performance llama.cpp fork with better CPU and hybrid GPU/CPU performance, SOTA quantization types, first-class Bitnet support, and improved DeepSeek performance via MLA, FlashMLA, and fused MoE operations. MIT licensed.
xLLM - High-performance inference engine optimized for Chinese AI accelerators (Cambricon MLU, Hygon DCU, Huawei Ascend). Features service-engine decoupled architecture with elastic scheduling, PD disaggregation, and global KV cache management. Powers JD.com's core retail businesses. Apache 2.0 licensed.
Mooncake - Production-grade serving platform for Kimi (Moonshot AI). Features distributed KV cache pool with intelligent offloading, prefill/decode disaggregation, and cross-instance KV reuse. Integrated with vLLM, SGLang, and TensorRT-LLM. Apache 2.0 licensed.
gemma.cpp - Lightweight, standalone C++ inference engine for Google's Gemma models. Optimized for on-device deployment with minimal dependencies and efficient memory usage. Apache 2.0 licensed.
FlashInfer - Kernel library for LLM serving. High-performance CUDA kernels for attention, sampling, and matrix multiplication. Powers vLLM, SGLang, and other inference engines with optimized GPU kernels. Apache 2.0 licensed.

Inference Kernels & Runtime Primitives

DeepEP - Communication library for efficient expert-parallel training/inference pipelines, reducing MoE cross-device communication overhead. Apache 2.0 licensed.
FlashKDA - High-performance CUTLASS-based Kimi Delta Attention CUDA kernels for linear attention mechanisms. MIT licensed.
DeepGEMM - Clean, high-performance FP8 GEMM kernels with fine-grained scaling for modern inference workloads. Apache 2.0 licensed.
RAFT - CUDA-accelerated algorithms and ANN building blocks for high-performance similarity search, clustering, and matrix learning workloads.
ThunderKittens - CUDA tile primitives and kernel templates for accelerating transformer attention blocks. MIT licensed.
tt-metal - TT-Metalium + TT-NN operator stack for building and optimizing kernels on Tenstorrent AI accelerators. Apache 2.0 licensed.
mini-sglang - Compact implementation of SGLang designed to demystify modern LLM serving systems. Educational yet production-quality with RadixAttention, continuous batching, and speculative decoding. MIT licensed.

Quantization, Distillation & Optimization

bitsandbytes - 8-bit and 4-bit optimizers + quantization.
Optimum - Hardware-specific acceleration and quantization.

4. Agentic AI & Multi-Agent Systems

Frameworks and platforms for building agent-based systems and multi-agent workflows.

Single-Agent Frameworks

AutoGPT - The original autonomous AI agent framework that sparked the agent revolution. Vision of accessible AI for everyone with modular agent architecture, benchmark testing, and forge-based agent building. 183k+ stars.
LangGraph - Stateful, controllable agent orchestration.
CrewAI - Role-based agent framework.
AutoGen (AG2) - Flexible multi-agent conversation framework.
DSPy - Framework for programming language model pipelines with modules, optimizers, and evaluation loops.
Semantic Kernel - SDK for building and orchestrating AI agents and workflows across multiple programming languages.
smolagents - Lightweight agent framework centered on tool use and code-executing workflows.
LangChain - Foundational library for agents, chains, and memory.
Neuron AI - PHP Agentic Framework for building production-ready AI driven applications. Connect components (LLMs, vector DBs, memory) to agents that can interact with your data. MIT licensed.
II-Agent (Intelligent Internet) - New open-source framework to build and deploy intelligent agents with support for Claude, Gemini, and OpenAI models. Apache 2.0 licensed.
Hermes Agent (NousResearch) - The agent that grows with you. Autonomous server-side agent with persistent memory that learns and improves over time.
Strands Agents - Model-driven approach to building AI agents in just a few lines of code. Multi-agent systems, autonomous agents, and streaming support with built-in MCP. Apache 2.0 licensed.
Agno - Build, run, and manage agentic software at scale. High-performance framework for multi-agent systems with memory, knowledge, and tools.
Upsonic - Agent framework for fintech and banking with built-in MCP support, guardrails, and tool server architecture.
VoltAgent - TypeScript-first AI agent engineering platform with memory, RAG, workflows, MCP integration, and voice support.
PocketFlow - 100-line minimalist LLM framework for building agent workflows. Lightweight, extensible architecture for tool use and autonomous task execution.
Agent Development Kit (Google) - Code-first Python toolkit for building sophisticated AI agents with multi-agent orchestration, built-in evaluation, and flexible deployment. Model-agnostic with tight Google ecosystem integration. Apache 2.0 licensed.
PydanticAI - Type-safe AI agent framework from the creators of Pydantic. Model-agnostic with 20+ providers, built-in observability via Logfire, MCP/A2A protocol support, and YAML/JSON agent definitions. MIT licensed.
Griptape - Modular Python framework for AI agents and workflows with chain-of-thought reasoning, tools, and memory. Enforces structures like sequential pipelines and DAG-based workflows for predictable AI systems. Apache 2.0 licensed.
Langroid - Harness LLMs with multi-agent programming. Mature tool calling system based on Pydantic, supports hundreds of LLM providers including OpenAI and local servers. Built for robust agent behavior in real-world use cases. MIT licensed.
Octomind - Model-agnostic AI agent runtime written in Rust with specialist agents, MCP support, multiple provider integrations, and zero-config setup. Apache 2.0 licensed.
Marvin - Python framework for structured outputs and agentic AI workflows. Simplifies LLM interactions with type-safe interfaces, automatic schema generation, and built-in observability. From the creators of Prefect. Apache 2.0 licensed.
Burr - Apache incubating framework for building stateful AI applications (chatbots, agents, simulations). Monitor, trace, persist, and execute on your own infrastructure with built-in UI and pluggable memory. Apache 2.0 licensed.
KaibanJS - JavaScript-native framework for building and managing multi-agent systems with a Kanban-inspired approach. Visual task board for AI agents with real-time collaboration features. MIT licensed.
Jido - Autonomous agent framework for Elixir. Built for distributed, autonomous behavior and dynamic workflows with actor-model concurrency. Apache 2.0 licensed.
Flue - Programmable TypeScript harness and sandbox agent framework for building autonomous workflows and agents. Apache 2.0 licensed.
Agent-Native - TypeScript-first framework for building agent-first applications featuring shared database state, real-time multiplayer editing, and action-driven tools. ISC licensed.
rlm - General plug-and-play inference library for Recursive Language Models (RLMs) that programmatically execute sub-LM calls inside isolated code sandboxes.
Page Agent - JavaScript-native, in-page GUI agent framework for controlling web interfaces with natural language without screenshots or external browser automation. MIT licensed.
Computer (Cloudflare) - Virtual filesystem inside a Durable Object providing sandboxed execution environments for AI agents. MIT licensed.
Embabel - Agent framework for the JVM written in Kotlin with dynamic goal-oriented planning and Spring Boot integration. Apache 2.0 licensed.
Ouroboros - Self-hosted general-purpose agent with durable identity and memory, specialist subagent coordination, and reviewed changes to its own implementation. MIT licensed.

Multi-Agent Orchestration

ChatDev - Multi-agent software development framework where AI agents collaborate as programmers, designers, and testers to build software. Apache 2.0 licensed.
CAMEL - First and best multi-agent framework for building scalable agent systems. Apache 2.0 licensed with extensive tooling for agent communication and task automation.
DeepAgents - Batteries-included LangChain agent harness for building and running structured multi-agent workflows with reusable runtime patterns.
EvoFlux - Local-first desktop workspace for coordinating AI agent teams across conversations, files, terminal, browser, Git, memory, plugins, and verification.
Swarms - Bleeding-edge enterprise multi-agent orchestration.
Mastra - TypeScript-first agent framework with built-in RAG, workflows, tool integrations, observability and observational memory.
Nika - Workflow engine for AI where agent work is captured as reviewable .nika.yaml DAG files, statically checked before execution (schema, permits, cost floor) with tamper-evident traces after. Local-first (Ollama, llama.cpp, vLLM), MCP client and server. AGPL-3.0 licensed.
Deer-Flow (ByteDance) - Open-source long-horizon SuperAgent harness that researches, codes, and creates. Handles tasks from minutes to hours with sandboxes, memories, tools, skills, subagents, and message gateway.
OpenAI Agents SDK - Production-ready lightweight framework for multi-agent workflows. The evolution of Swarm with enhanced orchestration capabilities and enterprise-grade features.
Symphony - Turns project work into isolated, autonomous implementation runs. Monitors work boards, spawns agents to handle tasks, and provides proof of work including CI status, PR reviews, and walkthrough videos. Engineering preview for managing work instead of supervising coding agents. Apache 2.0 licensed.
Paperclip - AI agent company and orchestration framework with 55K+ stars. MIT licensed.
AgentScope - Alibaba's production-ready multi-agent framework with 23K+ stars. Features built-in MCP and A2A support, message hub for flexible orchestration, and AgentScope Runtime for production deployment.
Microsoft Agent Framework - Microsoft's official framework combining AutoGen's agent abstractions with Semantic Kernel's enterprise features. Supports Python and .NET with graph-based workflows.
NarraNexus - A ready-to-run AI agent team workspace whose agents remember, collaborate, and use tools from day one. Apache 2.0 licensed.
Agency Swarm - Reliable multi-agent orchestration framework built on top of the OpenAI Assistants API with organizational structure modeling.
elizaOS - Autonomous multi-agent framework for building and deploying AI-powered applications. Features Discord/Telegram/Farcaster connectors, RAG support, and a modern web dashboard.
OpenAgents - AI Agent Networks for Open Collaboration. Platform for building collaborative multi-agent systems with shared knowledge and distributed task execution. Apache 2.0 licensed.
Hive (Aden) - Production-grade multi-agent orchestration framework with 10K+ stars. Apache 2.0 licensed.
Agent Squad (AWS Labs) - Flexible multi-agent orchestration framework with intelligent intent classification and context management. Supports Python and TypeScript with pre-built agents for Bedrock, Lex, and custom integrations. Apache 2.0 licensed.
DeepResearchAgent - Hierarchical multi-agent system for deep research tasks with automated task decomposition and execution across complex domains.
Composio Agent Orchestrator - Agentic orchestrator for parallel coding agents. Plans tasks, spawns agents, and autonomously handles CI fixes, merge conflicts, and code reviews. MIT licensed.
Open Multi-Agent - MIT-licensed TypeScript-native orchestration framework that plans multi-agent task DAGs at runtime, with approval gates, tracing, evaluation, checkpoints, and resumable execution in your own environment.
BeeAI Framework (IBM) - Production-ready multi-agent framework in Python and TypeScript. Features workflow orchestration, ACP/MCP protocol support, and deep watsonx integration. Part of Linux Foundation AI & Data program.
AI Town - Deployable starter kit for building virtual towns where AI characters live, chat and socialize. Inspired by Stanford's Generative Agents research with persistent agent memory and social interactions. MIT licensed.
Conductor OSS - Event-driven agentic orchestration platform providing durable and resilient execution engine for applications and AI agents. Battle-tested at Netflix, Tesla, LinkedIn, and J.P. Morgan with 30K+ stars. Apache 2.0 licensed.
A2A Protocol - Agent2Agent (A2A) open protocol enabling communication and interoperability between opaque agentic applications. Donated to Linux Foundation by Google with 50+ technology partners. Apache 2.0 licensed.
777genius/agent-teams-ai - Open-source desktop app for coordinating autonomous coding-agent teams with inter-agent messaging, Kanban task management, and code review across Claude Code, Codex, OpenCode, Cursor, Grok, GitHub Copilot, Kiro, Z.AI, MiniMax, Kimi, and 75+ LLM providers.
Panniantong/Agent-Reach - Reusable search and web-ingestion layer for AI agents spanning Reddit, X/Twitter, YouTube, GitHub, Bilibili and more through one CLI.
xerrors/Yuxi - Self-hosted multi-tenant agent harness combining retrieval, knowledge graph grounding, and workflow orchestration for production teams.
Sim Studio - Open-source AI workspace for building, deploying, and orchestrating AI agents. Visual canvas with 1000+ integrations, multi-framework support (Agno, OpenAI, LangChain, Google ADK), and self-hosted or cloud deployment. Apache 2.0 licensed.
2FastLabs Agent Squad - Flexible, lightweight open-source framework for orchestrating multiple AI agents to handle complex conversations with parallel execution capabilities. Apache 2.0 licensed.
SIA - Self-improving framework that orchestrates meta, target, and feedback agents to autonomously optimize the performance of AI models and agents on benchmark tasks. MIT licensed.
Council of High Intelligence - Multi-agent deliberation framework that routes specialized personas across different LLM providers to debate topics and reach consensus.
Gas Town - Multi-agent workspace manager and orchestration system for Claude Code and other coding agents with persistent work tracking, mailboxes, and automated merge queues. MIT licensed.
Agentlas OS Agent Operation Environment (AOE) - Local-first, model-agnostic environment for building, borrowing, and orchestrating specialist agents and teams across supported LLM hosts, with owner-scoped packages and host-enforced permissions. Apache 2.0 licensed.
Open Deep Research - Open-source deep research assistant that orchestrates language models and search tools to run multi-step web research and compile reports. MIT licensed.
Claudexor - Local-first control plane for routing coding work across Claude Code, Codex, Cursor, and OpenCode, with quota-aware rotation across multiple Claude/Codex subscription profiles, shared thread context, best-of-N runs, and cross-model review.
MiroFish - Multi-agent swarm intelligence engine that constructs parallel digital environments with autonomous agents to simulate and predict real-world outcomes. AGPL-3.0 licensed.
Maka - Apache-incubating local-first AI-agent workspace with durable append-only agent execution history, licensed under Apache-2.0.
Ruflo - Multi-agent orchestration and meta-harness for coordinating agent swarms with shared memory, licensed under the MIT license.
munder-difflin - MIT-licensed desktop multi-agent workspace and harness for coordinating local coding agents across multiple LLM backends.
fractal - Hierarchical coding-agent orchestrator with recursive delegation, per-node Git worktrees, configurable limits, persistent SQLite state, and live terminal monitoring and steering.
AX (Google) - Declarative, high-throughput agent orchestration runtime for sandboxed agent workloads at cluster scale. Apache 2.0 licensed.
Strands Agent Harness - Production SDK and harness runtime for building, monitoring, and controlling end-to-end AI agent lifecycles in Python and TypeScript. Apache 2.0 licensed.
StarNet - Local-first desktop multi-agent harness with persistent workspaces, capability-scoped tools, agent memory, budgets, schedules, and live runtime visualization. MIT licensed.
OpenRig - Local multi-agent harness for defining persistent Claude Code and Codex teams, with YAML topologies, tmux sessions, durable state, and a shared terminal UI. Apache-2.0 licensed.
Raven - Multi-agent harness whose host agent plans complex tasks as DAGs and delegates them to built-in research, coding, design, and on-call agents or to third-party agents over ACP, CLI, or OpenAI-compatible APIs, with cross-session memory and an experimental self-evolution loop that installs only verified changes. Apache 2.0 licensed.
5dive - Self-hosted CLI that runs a team of AI agents on one Linux server, each its own Linux user running Claude Code, Codex or another official agent CLI as a systemd service, sharing an org chart and a backlog. MIT licensed.

Agent Protocols & Standards

Agent Gateway - Next-generation proxy and routing layer for AI agents and MCP servers, with Kubernetes-native transport, protocol interoperability, and service-mesh-style isolation for reliable agent infrastructure. Apache 2.0 licensed.
Agent Governance Toolkit - Policy, safety, and execution controls for autonomous AI agents, including governance guardrails, sandboxing, and reliability checks. Apache 2.0 licensed.
Microsandbox - Fast, local-first microVM runtime and library for isolated execution of AI agents and untrusted workloads. Apache 2.0 licensed.
Agent Substrate - Kubernetes-native execution runtime for high-density, stateful AI-agent sandboxes with actor lifecycle management, suspend/resume, and workload routing. Apache-2.0 licensed.
DESIGN.md (Google) - A format specification for describing visual identity to coding agents, combining YAML tokens and markdown prose to give agents a structured understanding of design systems. Apache 2.0 licensed.
Agent Skills - Standardized specification and document format for bundling and progressively loading AI agent capabilities, instructions, scripts, and resources. Apache 2.0 licensed.
FastMCP - A Python framework for building Model Context Protocol (MCP) servers and clients with automatic schema generation and validation. Apache 2.0 licensed.

Agent Context, Memory & Knowledge

OpenViking - Open-source context database for AI agents that unifies agent memory, knowledge RAG, and skills as a virtual filesystem. AGPL 3.0 licensed.
LWC (Local Wiki CLI) - Local-first, source-grounded project memory for coding agents with bounded MCP retrieval, citations and provenance, atomic changesets, and optional document and code knowledge graphs. Apache 2.0 licensed.
Obsidian Agent Skills - Agent skills and open-format tooling for Obsidian vaults, Markdown, Bases, and JSON Canvas compatible with Claude Code, Codex, and OpenCode. MIT licensed.
claude-obsidian - MIT-licensed Obsidian and Claude Code integration that organizes Markdown vaults with local indexing and links.
Cangjie Skill - Pipeline to distill books, videos, and podcasts into structured, executable agent skills using verification and testing workflows. MIT licensed.
Hexis - Git-backed platform for managing and sharing skills, tools, and context across AI agents through a remote MCP server. Apache 2.0 licensed.
Codegraph - Local pre-indexed code knowledge graph for coding agents to reduce token usage and redundant tool calls across Claude, Codex, and other agents.
Wenlan - Local source-backed AI knowledge base and LLM wiki for coding agents, with Markdown pages, hybrid retrieval, knowledge graphs, CLI workflows, and MCP integrations. Apache 2.0 licensed.
Gortex - Local-first code knowledge graph and code intelligence engine built in Golang with multi-repository support and real-time graph actualisation. Built for AI coding agents aiming to expose only the needed information, reducing token usage by up to 50x. Works natively with Claude, Codex, Hermes, and other agents.
Graphify - AI coding assistant skill that indexes codebases, databases, and documents into a queryable knowledge graph for coding agents. MIT licensed.
Workspai - Open-source workspace intelligence CLI that models polyglot workspaces, builds proof-backed knowledge graphs, runs health and release gates, and generates bounded context for AI coding agents. MIT licensed.
Headroom - Context compression proxy for tool outputs, logs, and RAG chunks, reducing token pressure while preserving intent for AI agents.
llmtrim - Self-hosted Rust proxy, MCP server, CLI, and library that compresses LLM prompts, tool outputs, and replies to cut token usage, quality-gated so it never raises your bill. MPL-2.0 licensed.
MailFathom - Security-first, self-hosted email knowledge system that synchronizes IMAP mailboxes into user-owned PostgreSQL for lexical and semantic retrieval, cited answers, local-model operation, and MCP access. AGPL-3.0 licensed.
MemPalace - High-performance, benchmarked AI memory system for persistent recall and retrieval in long-horizon autonomous workflows.
Supermemory - Memory engine and API designed for long-lived AI agents to store, retrieve, and reuse long-horizon context with low latency.
Tree Ring Memory - Framework-agnostic, local-first memory lifecycle for AI agents with a Rust CLI, SQLite/FTS recall, redaction, forgetting, audit checks, consolidation, and agent-skill guidance. MIT licensed.
deja - Memory layer over the session transcripts coding agents already write to disk — Claude Code, Codex, Cursor, opencode, Zed and more — including sessions from before it was installed; local BM25 recall over them with no LLM or embeddings, MCP tools, and credentials redacted at index time. MIT licensed.
Claude Mem - Persistent agent-memory system that captures session observations, summarizes them, and retrieves relevant context across coding-agent sessions. Apache-2.0 licensed.
codebase-memory-mcp - High-performance C-based codebase intelligence engine and MCP server that indexes repositories into local type-resolved knowledge graphs. MIT licensed.
book-to-skill - CLI tool that distills technical books, documents, and reference materials into structured, on-demand agent skills. MIT licensed.
AI Memory - Rust-native long-term memory server and MCP client for agent coding CLIs, featuring Karpathy-style LLM wiki compilation, FTS5 recall, and cross-agent session handoffs. MIT licensed.
Busabase - Open-source database and workspace that gives AI agents structured data, durable knowledge, docs, skills, and apps through MCP, OpenAPI, CLI, and coding-agent skills; material writes can remain reviewable ChangeRequests before becoming canonical. MIT licensed.
ThreadShelf - Local-first archive and semantic search for AI conversation histories across multiple providers, with local embeddings, LanceDB storage, complete thread retrieval, HTTP and CLI access, and a stdio MCP server.

Autonomous Coding Agents

BitFun - Open-source coding agent with a Rust runtime, desktop and CLI interfaces, self-hosted remote access, and support for custom tools and skills.
Free Claude Code - Multi-provider proxy and launcher for Claude Code, Codex, and Pi with a local Admin UI to route coding agents to 31+ cloud and local LLM backends. MIT licensed.
Background Agents - Open-source background coding agent system inspired by Ramp's Inspect, supporting file and environment snapshots, cron-based automation, and multi-provider models. MIT licensed.
OpenHands (ex-OpenDevin) - Full-featured open-source AI software engineer.
HEXStrike AI - MCP-powered coding-focused cybersecurity agent framework for automated pentesting and bug-hunting workflows.
Darkmoon - Open-source autonomous AI pentest platform and MCP host orchestrating 80+ offensive tools via per-tech offensive sub-agents, with Active Directory and Kubernetes support. GPL-3.0 licensed.
VulnClaw - Autonomous penetration testing agent utilizing Model Context Protocol (MCP) toolchains, blackboard state space search, and structured reasoning.
Goose - Extensible on-machine AI agent for development tasks.
OpenShell (NVIDIA) - Safe and private runtime for autonomous AI agents with policy-driven execution boundaries and CLI integration.
CodeGraph - Pre-indexed local code knowledge graph for Claude Code, Codex, Gemini, and other coding agents to reduce context churn and token spend in developer workflows.
OpenCode - Terminal-native autonomous coding agent.
ECC - Performance-oriented agent harness for coding agents with skills, memory, and security-aware orchestration across Claude Code, Codex, and more.
oh-my-pi - Terminal coding agent with hash-anchored edits, subagents, LSP, browser integrations, and terminal-native workflows.
Pi (earendil-works) - Modular agent toolkit with terminal-first CLI, unified model/provider layer, and runtime integrations for coding workflows and TUI/web UIs.
Aider - Command-line pair-programming agent.
Pi (badlogic) - Terminal coding agent with hash-anchored edits, LSP integration, subagents, MCP support, and package ecosystem.
Mistral-Vibe (Mistral) - Minimal CLI coding agent by Mistral. Lightweight, fast, and designed for local development workflows.
Nanocoder (Nano-Collective) - Beautiful local-first coding agent running in your terminal. Built for privacy and control with support for multiple AI providers via OpenRouter.
Gemini CLI (Google) - Open-source AI agent that brings Gemini's power directly into your terminal. Supports code generation, shell execution, and file editing with full Apache 2.0 licensing.
Archon - Workflow engine for deterministic AI coding agents. Define development processes as YAML workflows (planning → implementation → validation → review → PR) with isolated Git worktrees for parallel execution. MIT licensed.
mini-SWE-agent - Lightweight coding agent for repository and issue-fixing workflows, designed for simple agentic software engineering experiments.
Kilo Code - Open-source agentic coding assistant with IDE workflows, tool use, and support for local or OpenAI-compatible models.
Open SWE - Asynchronous coding agent from the LangChain ecosystem for background software engineering tasks.
Letta Code - Memory-first coding harness designed for long-lived agents that learn from experience. Persistent agents with portable memory across models (Claude, GPT, Gemini, GLM, Kimi). CLI and desktop app for macOS, Windows, and Linux. Apache 2.0 licensed.
LoopTroop - Local-first AI coding workspace that orchestrates multi-model planning councils, git worktrees, and task loops. MIT licensed.
gptme - Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web. Make your own persistent autonomous agent on top. MIT licensed.
Superpowers - Composable skills framework and software development methodology for coding agents, structuring processes like planning, test-driven development, and code review.
Agent Skills - Production-grade engineering skills and quality gates for AI coding agents, packaging developer workflows like spec refinement, planning, and testing.
Harness - Team-architecture factory for C‍laude Code that designs domain-specific agent teams, defines specialized agents, and generates their skills. Apache 2.0 licensed.
jcode - Performance-oriented, memory-efficient coding agent harness built for multi-session workflows and infinite customizability. MIT licensed.
firstmate - Agent distro for running a crew of autonomous coding agents in isolated Git worktrees across visible terminal session backends. MIT licensed.
DeepSeek-Reasonix - DeepSeek-native AI coding agent for the terminal, designed around prefix-cache stability. MIT licensed.
gstack - Multi-specialist agent skills framework for Claude Code and coding agents that structures development workflows into planning, design, QA, and release phases. MIT licensed.
taste-skill - Anti-slop frontend design skills for coding agents that improve layout, typography, and design system alignment during generation. MIT licensed.
Agents CLI (Google) - CLI and skills that turn coding assistants into experts at creating, evaluating, and deploying AI agents on Google Cloud. Apache 2.0 licensed.
Claude Code Skills & Plugins - Modular instruction packages, custom commands, and utility scripts for Claude Code, Gemini CLI, Cursor, and other AI coding agents. MIT licensed.
AG Kit - Antigravity-first agent engineering kit featuring rules, skills, persistent memory, MCP guidance, and a native safety hook. MIT licensed.
Prime Agent - Self-improving recursive language model (RLM) agent for coding workflows and autonomous tasks. MIT licensed.
Agent Skills (Google) - Official collection of Agent Skills for Google Cloud and Google developer platforms, extending AI coding agents with product and technology workflows. Apache 2.0 licensed.
Agent Skills (Anthropic) - Official collection of Agent Skills and reference implementations for Claude Code, Claude API, and AI agents. Apache 2.0 licensed.
Superagent - macOS desktop app giving Claude Code and Codex a real browser