pplx-garden

Perplexity AI open source garden for inference technology

Projects

fabric-lib

RDMA TransferEngine, P2P MoE dispatch/combine kernel

• Docs: docs/fabric-lib.md
• MLSys'26 paper: fabric-lib: RDMA Point-to-Point Communication for LLM Systems
• Blog Post: RDMA Point-to-Point Communication for LLM Systems
• Blog Post: Enabling Trillion-Parameter Models on AWS EFA
• Blog Post: Weight Transfer for RL Post-Training in under 2 seconds
• Blog Post: Disaggregated Prefill and Decode

pplx-unigram

Unigram tokenizer encoder

• Docs: docs/unigram.md
• Blog Post: Improving Unigram Tokenizer CPU Performance

lily

Rust and Metal inference server for Qwen3.6-35B-A3B on Apple Silicon. Lily provides greedy text generation through a minimal OpenAI-compatible HTTP API.

• Docs: lily/README.md
• Blog Post: Optimizing On-Device Inference for Apple Silicon
• License: Apache-2.0, with third-party notices in lily/NOTICE

Directory Structure

• fabric-lib/: RDMA TransferEngine library
• lily/: Metal LLM inference for Apple Silicon
• p2p-all-to-all/: P2P MoE All-to-All implementation
• pplx-unigram/: Unigram tokenizer encoder
• python-ext/: Python extension module from Rust code
• python/pplx_garden/: Python code for the pplx_garden package
• rust/: Rust utility libraries