A source-backed collection of papers, models, datasets, benchmarks and tools for agents that use computers.
English · 简体中文
Search the website · Papers · Topics · Data · Agent guide · Contribute
If this collection is useful, a star on GitHub helps others find it.
Curated entries were reviewed against a first-party source snapshot; records awaiting review have their bibliographic identity checked; candidates are discovered identifiers awaiting resolution. What “verified” means · Coverage ledger
| Website reconstruction | Browser agents |
|---|---|
Can AI agents rebuild websites through browser exploration? ★ Star on GitHub |
Open-source benchmark for browser AI agents on daily tasks. ★ Star on GitHub |
Featured: Awesome Reverse Engineering Anything — a collection of tools, case studies and research for reverse-engineering websites, games, apps, native binaries, firmware and protocols.
| Your task | Start here | What you can find |
|---|---|---|
| Find papers | Research library · Topics · Paper–asset connections | Reviewed papers by year and topic, with original sources and the code, datasets and models linked to them through reviewed primary evidence. |
| Choose a benchmark | Benchmark guide | Platforms, tasks, evaluators, and versions to compare. |
| Find training data | Training-data guide | Supervision, splits, count units, and data licenses. |
| Build an agent | Agents · Frameworks | Implementation entry points, supported platforms, and papers. |
| Use RAG or an agent | Agent guide · Data hub | Complete records or retrieval documents with review states and sources. |
Local examples (after cloning; Python 3.11+, no dependencies required)
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind paper --status curated --limit 0 --json
python3 scripts/catalog.py search --kind benchmark --environment web --status curated --limit 0 --json
python3 scripts/catalog.py search --kind dataset --environment android --status curated --limit 0 --json
python3 scripts/catalog.py search --kind framework --status curated --limit 0 --json
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/catalog.py get arxiv:2404.07972 --related --json2,170 curated resources · 1,673 reviewed papers and surveys · 104 discovery documents
Evidence through 2026-11-07T05:19:53Z. 453 records awaiting review have verified bibliographic metadata, and 2,062 discovered candidates await resolution. 1,308 scope exclusions retain their original metadata in the archive.
| Browse | What you find | Entries |
|---|---|---|
| Research by topic | Reviewed papers grouped by topic | 1,645 |
| Full bibliography | All paper and survey records, including those awaiting review | 2,126 |
| Surveys | Surveys of computer-use agent research | 28 |
| Learning resources | Courses and tutorials | 2 |
| Datasets | Training and evaluation data for GUI agents | 86 |
| Benchmarks | Evaluation environments and task suites | 91 |
| Models and checkpoint releases | Checkpoints and model families | 98 |
| Agents | Open implementations that act on a computer | 49 |
| Frameworks | Code for building and evaluating agents | 87 |
| Infrastructure | Sandboxes, VMs, device farms and environments | 16 |
| Tools | Evaluation harnesses, recorders and other utilities | 32 |
| Products | Computer-use products from vendors and projects | 20 |
| Publishers and brands | Publishers and brands connected to cataloged resources | 16 |
By platform: Web 858 · Desktop 364 · Windows 68 · macOS 36 · Linux 57 · Android 350 · iOS 26 · HarmonyOS 4 · Cross-platform 206
Coverage spans web, desktop, Android, iOS, Windows, macOS, Linux, and cross-platform work. Topics include grounding, screen understanding, planning, memory, long-horizon workflows, reinforcement learning, data generation, safety, robustness, and evaluation.
Download the catalog, feeds and RAG documents; the data hub also lists sources, review states and schemas.
- llms.txt - Navigation file for language models.
- llms-full.txt - Curated annotations in Markdown.
- Data hub - Catalog, RAG documents, citations, topics, comparisons, sources and schemas in six groups.
- Catalog JSONL - Complete catalog with one resource per line.
- Download manifest - Export counts, byte sizes and SHA-256 checksums.
- Atom feed - Research updates as an Atom feed.
- Hugging Face dataset - Read-only mirror; metadata_only and github_candidates are unreviewed.
- Agent and RAG guide - Guide for agents and RAG systems.
Clone the complete catalog:
git clone https://github.com/reacher-z/Awesome-CUA.gitAgent instructions · Retrieval documents
Discovery is scheduled every hour at minute 7 UTC. Actual scan results · Schedule and safeguards.
Read the research survey — Methods, evaluation, human agency, and reusable research assets. English working manuscript with a versioned bibliography and reading register.
The most-mentioned curated resources per category: the count is how many other cataloged resources mention each one in their reviewed first-party source (README, model or dataset card). It reflects connectedness inside this catalog. Every category page lists all reviewed entries. Surveys come first, newest first.
Show the highlights tables
Start with the newest surveys · all 28 →
| Date | Survey | Environments |
|---|---|---|
| 2026‑10‑21 | Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use | unspecified |
| 2026‑10‑14 | Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory | unspecified |
| 2026‑09‑02 | Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization | cross-platform |
| 2026‑08‑30 | A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security | cross-platform |
| 2026‑08‑10 | Software Engineering for and with GUI Agent | web |
Benchmarks · all 91 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| OSWorld | 22 | desktop, linux, windows, macos | 2404.07972 (2024) | code Apache-2.0 | Execution-based evaluation of agents performing tasks in real desktop environments. |
| WebArena | 15 | web | 2307.13854 (2023) | code Apache-2.0 | Self-hosted web environments and tasks for evaluating autonomous web agents. |
| BrowserGym | 9 | web | 2412.05467 (2024) | code Apache-2.0 | Unified interfaces for browser-agent tasks and benchmark environments. |
| ScreenSpot Pro | 8 | desktop, windows, macos, linux | — | code MIT | GUI grounding benchmark focused on professional high-resolution interfaces. |
| VisualWebArena | 7 | web | 2401.13649 (2024) | — | Visually grounded web tasks extending the WebArena evaluation setting. |
| AndroidWorld | 6 | android | 2405.14573 (2024) | code Apache-2.0 | Android environment with programmatic task evaluation for mobile agents. |
Models · all 98 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| SeeClick | 9 | cross-platform | 2401.10935 (2024) | code Apache-2.0 | GUI grounding model trained to locate visual interface elements. |
| Aguvis | 5 | cross-platform | 2412.04454 (2024) | — | Vision-based GUI agent models with unified action representations. |
| UI-TARS | 5 | cross-platform | 2501.12326 (2025) | code Apache-2.0 | Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use. |
| OpenCUA | 3 | desktop | 2508.09123 (2025) | code MIT | Open computer-use research stack including models and training resources. |
| OS-Atlas | 3 | cross-platform | 2410.23218 (2024) | code Apache-2.0 | OS-Atlas GUI action models, including grounding bases fine-tuned from InternVL2-4B and Qwen2-VL-7B-Instruct that accept any image size and output 0-1000 relative coordinates. |
| ShowUI | 3 | cross-platform | 2411.17465 (2024) | code Apache-2.0 | Lightweight GUI-agent vision-language-action model from Show Lab and Microsoft, released with UI-guided token selection, training code, the ShowUI-web and ShowUI-desktop datasets and vLLM inference. |
Datasets · all 86 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| ScreenSpot-Pro data | 4 | desktop, windows, macos, linux | 2504.07981 (2025) | data mit | High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization. |
| GUI-Odyssey | 3 | android | 2406.08451 (2024) | — | Human-demonstrated Android episodes for cross-app navigation, spanning six task types, 212 apps and about 1,400 app combinations, for training and evaluating agents. |
| GUIAct | 3 | web, android | 2406.11317 (2024) | — | Web and Android GUI action-instruction data, with single-step, multi-step and smartphone partitions. |
| GUIChat | 3 | unspecified | 2406.11317 (2024) | — | Conversational GUI supervision with text-rich images and bounding boxes, organized as single-turn questions and multi-turn dialogues. |
| GUIEnv-local | 3 | web | 2406.11317 (2024) | — | The released local-supervision portion of GUIEnv, pairing webpage regions and text for OCR and GUI grounding. |
| OpenMobile-Data-v2 | 3 | android, web | 2610.14157 (2026) | — | Mobile-agent supervision from emulator and simulator trajectories, plus executable tasks with starting states and success checks for reinforcement learning. |
Frameworks · all 87 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| Browser Use | 10 | web | — | code MIT | Open-source browser agent library that runs locally with any LLM against a local or cloud browser, and also serves as a CLI for coding agents and as a driver for Claude's browser toolset. |
| AgentLab | 5 | web | 2412.05467 (2024) | code Apache-2.0 | Experiment framework for developing and evaluating browser agents. |
| Seg2Click | 5 | cross-platform | 2610.16234 (2026) | code MIT | Training and inference implementation that predicts an executable-region mask and derives a GUI click from that mask. |
| OpenMobile-2 code | 3 | android, web | 2610.14157 (2026) | code Apache-2.0 | Evaluation adapters and training guidance for OpenMobile-2 across Android emulators and browser-rendered mobile simulation. |
| UI-Genie | 3 | android | 2505.21496 (2025) | — | Training and evaluation code for a mobile-agent framework that iteratively generates trajectories using agent and reward-model feedback. |
| Computer Use OOTB | 2 | desktop | 2411.10323 (2024) | code Apache-2.0 | Gradio desktop GUI agent for Windows and macOS that needs no Docker, using Claude 3.5 Computer Use over the API or ShowUI and UI-TARS run locally, with multi-display and phone remote control. |
Agents · all 49 →
| Resource | Mentions | Environments | Paper | License | Description |
|---|---|---|---|---|---|
| Mobile-Agent | 9 | android, cross-platform | 2401.16158 (2024), 2406.01014 (2024), 2508.15144 (2025), 2602.16855 (2026) | code MIT | Mobile device agent family using visual interface interaction. |
| Agent S | 2 | desktop | 2410.08164 (2024), 2504.00906 (2025) | code Apache-2.0 | Simular's open-source framework for computer-use agents that click, type and scroll through desktop and web apps on macOS, Windows and Linux; its third generation, Agent S3, reports 72.6% on OSWorld. |
| SeeAct | 2 | web | 2401.01614 (2024) | — | Generalist web agent framework for large multimodal models such as GPT-4V, with a Playwright tool that runs agents on live websites, text-choice and set-of-mark grounding, and a crawler mode. |
| Agent E | 1 | web | 2407.13032 (2024) | code MIT | Browser automation agent on the AG2 framework that pairs a planner agent with a browser navigation agent to fill forms, search shops and work through sites, also callable over a local HTTP API. |
| AppAgent | 1 | android | 2312.13771 (2023) | code MIT | Multimodal agent that operates Android apps through taps and swipes without back-end access, building a reusable knowledge base from autonomous exploration or from watching human demonstrations. |
| Auto-GUI / Auto-UI | 1 | android | 2309.11436 (2023) | code Apache-2.0 | Screenshot-based mobile agent implementation with training and inference code for chain-of-action prediction. |
Newest editorial approvals of models, datasets, benchmarks, systems, and tools (newest first). New papers appear by date in the topic hubs.
Show the recently reviewed table
| Reviewed | Resource | Kind | Environments | Description |
|---|---|---|---|---|
| 2026‑11‑06 | web-eval-agent | tool | web | MCP server that launches a browser-use agent to run a web app, capture screenshots, console and network logs, and return a UX report to the coding agent. |
| 2026‑11‑06 | TestZeus Hercules | agent | web | Open-source testing agent that takes Gherkin features and executes them end to end on web applications through Playwright. |
| 2026‑11‑06 | TestDriver SDK | framework | desktop, web | Vitest plugin that spawns ephemeral devices and uses vision-based LLMs to build end-to-end tests of web apps, desktop apps and browser extensions. |
| 2026‑11‑06 | Tactile | tool | desktop, macos | Skill, protocol and tool layer that has agents operate software through accessibility semantics first, reading roles, names and states before reaching for screenshots. |
| 2026‑11‑06 | SeleniumBase | framework | web | Python browser automation framework for web crawling, end-to-end testing and scraping, with a recorder that generates tests and stealth options. |
| 2026‑11‑06 | Qwen Code | agent | desktop | Qwen Code is an open-source coding agent that supports desktop automation through computer-use tools alongside terminal, editor, web, and chat interfaces. |
| 2026‑11‑06 | puppeteer-extra | framework | web | A modular plugin framework for Puppeteer, maintained in a monorepo with plugins and related packages. |
| 2026‑11‑06 | Playwright MCP Server (ExecuteAutomation) | tool | web | Community MCP server that gives LLMs Playwright browser automation for interacting with web pages, taking screenshots, generating test code and running JavaScript. |
| 2026‑11‑06 | pi-computer-use | tool | desktop, macos, windows, linux | Pi extension that gives agents tools to find app windows, observe and search visible controls, click, type, scroll and wait for UI changes in desktop software. |
| 2026‑11‑06 | PhantomJS | tool | web | PhantomJS is a scriptable headless WebKit tool for automating web pages, capturing page contents, testing, and monitoring network activity. |
| 2026‑11‑06 | OpenManus | framework | web | OpenManus is an open-source agent framework that runs tasks from a terminal and integrates browser execution and screenshots through Browser Use. |
| 2026‑11‑06 | mcp-server-browser-use | tool | web | MCP server that wraps browser-use so that an MCP client can automate a real browser, with a CLI, a web dashboard and a deep research mode. |
Browse the interactive website to search and filter all resources, inspect source evidence, switch between English and Chinese, or download the complete catalog. Website development and publishing.
The English data hub and 中文数据中心 organize public downloads into six groups: complete catalog, RAG documents, citations, topics and comparisons, sources and review status, and schemas. Cloning still gives you the complete source catalog; the hub provides exports from its published snapshot.
The download manifest records each listed export's actual count and unit, scope, byte size, and SHA-256 fingerprint. Compare catalog_sha256 across manifests before combining files from different downloads.
Reviewed research · Computer-use agent guides · Static resource directory · Search and AI discovery
- Catalog JSONL: one resource per line, with stable IDs, typed fields, provenance, scoped licenses, and review status.
- Reference graph: outbound references from primary sources.
- Discovery registry and candidates: source snapshots and where each identifier was found.
- Review queue: unresolved evidence, unindexed discoveries, and proposals requiring editorial work. Discovered links reviewed as non-resources or as other addresses of indexed resources stay listed with a
resolutionand their reason. - JSON Schema and coverage JSON: machine-checkable contracts and coverage accounting.
- BibTeX and CSL JSON: reviewed-paper citations with recorded authors, dates, and versions. Each resource page also provides its own
record.json.
# Python 3.11+. Search and export require only the standard library.
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind model
python3 scripts/catalog.py search --kind dataset --environment android
python3 scripts/catalog.py search --kind product --status curated --json
python3 scripts/catalog.py get arxiv:2404.07972 --related --json
python3 scripts/catalog.py build
# Creates dist/catalog.sqlite and dist/catalog.json, with no network calls.
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/export_citations.py --format bibtex --output dist/citations.bib
# Add --include-metadata to include every active record, with its review state intact.Use SQLite to join resources, environment/topic facets, and reviewed relations. See query examples. data/catalog.jsonl is committed; SQLite and JSON array exports are reproducible local build products.
| State | What has been checked |
|---|---|
curated |
AI-assisted editorial review against a first-party source snapshot, with name, scope, summary, and taxonomy |
metadata_verified |
Bibliographic identity, title, authors, and recorded dates checked against retained arXiv or publication sources |
candidate |
A normalized URL or identifier was discovered |
A paper, repository, dataset release, model checkpoint, product, and publisher can be separate related resources. Model families and individual checkpoints are explicitly linked where reviewed.
Code, data, weights, and document licenses are separate. Missing values are null.
python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-dev.txt
.venv/bin/python scripts/catalog.py validate
.venv/bin/python scripts/catalog.py build --check
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/catalog.py build
node --test tests/web.test.mjs
.venv/bin/python scripts/build_site.pyThese commands match the checks that Catalog quality runs on every push and pull request, in the same order. They need Python 3.11+ and Node.js 24+, the engines version in package.json (CI uses Python 3.12 and Node.js 24). The web tests use only the built-in Node.js test runner, so nothing needs to be installed with npm. npm run test:web and npm run build:web are shortcuts for the last two commands.
Network discovery and verification are explicit commands; ordinary builds are offline. Changed primary snapshots require another editorial review before publication. Maintenance guide · Methodology · Data dictionary
The source registry credits the community collections and surveys used for discovery. Bibliographic records use retained arXiv, publisher, and author sources; publication observations preserve edition-specific titles, authors, and date precision. Selected assets are checked against publisher repositories, documentation, and model/dataset cards. Upstream READMEs, papers, model weights, and datasets are not redistributed here.
Original repository code and annotations use the MIT license. Linked resources retain their own terms.
To cite the catalog, use CITATION.cff or GitHub's "Cite this repository" button. The data changes with every scan, so pin the exact version you used. Each v* tag has a data release with versioned downloads built from the tagged commit: the catalog as JSONL, JSON and SQLite, the coverage ledger, topic and profile exports, and their SHA256SUMS. Cite the release tag, or record the commit from git rev-parse HEAD in your clone, or link files at that commit, such as https://github.com/reacher-z/Awesome-CUA/blob/<commit>/data/catalog.jsonl. Cutting a release.