Ying Sheng

Running large language models on a single GPU for throughput-oriented scenarios.

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.