Tony Lee

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.