← 开源
reacher-z

Awesome-CUA

Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.

ListsPaper collectionsDomain-specificPython
在 GitHub 打开
增长势头
+024 小时新增 Star0.0%
508
Star
137
Fork
4
贡献者
创建于 2026-10-16 · 更新于 2026-11-06 · 今日第 18743 名
主要开发者
reacher-zaazminu
README

A source-backed collection of papers, models, datasets, benchmarks and tools for agents that use computers.

English · 简体中文

Search the website · Papers · Topics · Data · Agent guide · Contribute

If this collection is useful, a star on GitHub helps others find it.

Curated entries were reviewed against a first-party source snapshot; records awaiting review have their bibliographic identity checked; candidates are discovered identifiers awaiting resolution. What “verified” means · Coverage ledger

Featured benchmarks

Website reconstruction Browser agents

Can AI agents rebuild websites through browser exploration?
★ Star on GitHub

Open-source benchmark for browser AI agents on daily tasks.
★ Star on GitHub

Featured: Awesome Reverse Engineering Anything — a collection of tools, case studies and research for reverse-engineering websites, games, apps, native binaries, firmware and protocols.

Find by task

Your task Start here What you can find
Find papers Research library · Topics · Paper–asset connections Reviewed papers by year and topic, with original sources and the code, datasets and models linked to them through reviewed primary evidence.
Choose a benchmark Benchmark guide Platforms, tasks, evaluators, and versions to compare.
Find training data Training-data guide Supervision, splits, count units, and data licenses.
Build an agent Agents · Frameworks Implementation entry points, supported platforms, and papers.
Use RAG or an agent Agent guide · Data hub Complete records or retrieval documents with review states and sources.
Local examples (after cloning; Python 3.11+, no dependencies required)
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind paper --status curated --limit 0 --json
python3 scripts/catalog.py search --kind benchmark --environment web --status curated --limit 0 --json
python3 scripts/catalog.py search --kind dataset --environment android --status curated --limit 0 --json
python3 scripts/catalog.py search --kind framework --status curated --limit 0 --json
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/catalog.py get arxiv:2404.07972 --related --json

2,170 curated resources · 1,673 reviewed papers and surveys · 104 discovery documents

Evidence through 2026-11-07T05:19:53Z. 453 records awaiting review have verified bibliographic metadata, and 2,062 discovered candidates await resolution. 1,308 scope exclusions retain their original metadata in the archive.

Explore

Browse What you find Entries
Research by topic Reviewed papers grouped by topic 1,645
Full bibliography All paper and survey records, including those awaiting review 2,126
Surveys Surveys of computer-use agent research 28
Learning resources Courses and tutorials 2
Datasets Training and evaluation data for GUI agents 86
Benchmarks Evaluation environments and task suites 91
Models and checkpoint releases Checkpoints and model families 98
Agents Open implementations that act on a computer 49
Frameworks Code for building and evaluating agents 87
Infrastructure Sandboxes, VMs, device farms and environments 16
Tools Evaluation harnesses, recorders and other utilities 32
Products Computer-use products from vendors and projects 20
Publishers and brands Publishers and brands connected to cataloged resources 16

By platform: Web 858 · Desktop 364 · Windows 68 · macOS 36 · Linux 57 · Android 350 · iOS 26 · HarmonyOS 4 · Cross-platform 206

Coverage spans web, desktop, Android, iOS, Windows, macOS, Linux, and cross-platform work. Topics include grounding, screen understanding, planning, memory, long-horizon workflows, reinforcement learning, data generation, safety, robustness, and evaluation.

Use the data

Download the catalog, feeds and RAG documents; the data hub also lists sources, review states and schemas.

Clone the complete catalog:

git clone https://github.com/reacher-z/Awesome-CUA.git

Agent instructions · Retrieval documents

Discovery is scheduled every hour at minute 7 UTC. Actual scan results · Schedule and safeguards.

Read the research survey — Methods, evaluation, human agency, and reusable research assets. English working manuscript with a versioned bibliography and reading register.

Highlights

The most-mentioned curated resources per category: the count is how many other cataloged resources mention each one in their reviewed first-party source (README, model or dataset card). It reflects connectedness inside this catalog. Every category page lists all reviewed entries. Surveys come first, newest first.

Show the highlights tables

Start with the newest surveys · all 28 →

Date Survey Environments
2026‑10‑21 Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use unspecified
2026‑10‑14 Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory unspecified
2026‑09‑02 Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization cross-platform
2026‑08‑30 A Systematic Survey of Agentic Skills: Architecture, Lifecycle, and Security cross-platform
2026‑08‑10 Software Engineering for and with GUI Agent web

Benchmarks · all 91 →

Resource Mentions Environments Paper License Description
OSWorld 22 desktop, linux, windows, macos 2404.07972 (2024) code Apache-2.0 Execution-based evaluation of agents performing tasks in real desktop environments.
WebArena 15 web 2307.13854 (2023) code Apache-2.0 Self-hosted web environments and tasks for evaluating autonomous web agents.
BrowserGym 9 web 2412.05467 (2024) code Apache-2.0 Unified interfaces for browser-agent tasks and benchmark environments.
ScreenSpot Pro 8 desktop, windows, macos, linux — code MIT GUI grounding benchmark focused on professional high-resolution interfaces.
VisualWebArena 7 web 2401.13649 (2024) — Visually grounded web tasks extending the WebArena evaluation setting.
AndroidWorld 6 android 2405.14573 (2024) code Apache-2.0 Android environment with programmatic task evaluation for mobile agents.

Models · all 98 →

Resource Mentions Environments Paper License Description
SeeClick 9 cross-platform 2401.10935 (2024) code Apache-2.0 GUI grounding model trained to locate visual interface elements.
Aguvis 5 cross-platform 2412.04454 (2024) — Vision-based GUI agent models with unified action representations.
UI-TARS 5 cross-platform 2501.12326 (2025) code Apache-2.0 Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use.
OpenCUA 3 desktop 2508.09123 (2025) code MIT Open computer-use research stack including models and training resources.
OS-Atlas 3 cross-platform 2410.23218 (2024) code Apache-2.0 OS-Atlas GUI action models, including grounding bases fine-tuned from InternVL2-4B and Qwen2-VL-7B-Instruct that accept any image size and output 0-1000 relative coordinates.
ShowUI 3 cross-platform 2411.17465 (2024) code Apache-2.0 Lightweight GUI-agent vision-language-action model from Show Lab and Microsoft, released with UI-guided token selection, training code, the ShowUI-web and ShowUI-desktop datasets and vLLM inference.

Datasets · all 86 →

Resource Mentions Environments Paper License Description
ScreenSpot-Pro data 4 desktop, windows, macos, linux 2504.07981 (2025) data mit High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization.
GUI-Odyssey 3 android 2406.08451 (2024) — Human-demonstrated Android episodes for cross-app navigation, spanning six task types, 212 apps and about 1,400 app combinations, for training and evaluating agents.
GUIAct 3 web, android 2406.11317 (2024) — Web and Android GUI action-instruction data, with single-step, multi-step and smartphone partitions.
GUIChat 3 unspecified 2406.11317 (2024) — Conversational GUI supervision with text-rich images and bounding boxes, organized as single-turn questions and multi-turn dialogues.
GUIEnv-local 3 web 2406.11317 (2024) — The released local-supervision portion of GUIEnv, pairing webpage regions and text for OCR and GUI grounding.
OpenMobile-Data-v2 3 android, web 2610.14157 (2026) — Mobile-agent supervision from emulator and simulator trajectories, plus executable tasks with starting states and success checks for reinforcement learning.

Frameworks · all 87 →

Resource Mentions Environments Paper License Description
Browser Use 10 web — code MIT Open-source browser agent library that runs locally with any LLM against a local or cloud browser, and also serves as a CLI for coding agents and as a driver for Claude's browser toolset.
AgentLab 5 web 2412.05467 (2024) code Apache-2.0 Experiment framework for developing and evaluating browser agents.
Seg2Click 5 cross-platform 2610.16234 (2026) code MIT Training and inference implementation that predicts an executable-region mask and derives a GUI click from that mask.
OpenMobile-2 code 3 android, web 2610.14157 (2026) code Apache-2.0 Evaluation adapters and training guidance for OpenMobile-2 across Android emulators and browser-rendered mobile simulation.
UI-Genie 3 android 2505.21496 (2025) — Training and evaluation code for a mobile-agent framework that iteratively generates trajectories using agent and reward-model feedback.
Computer Use OOTB 2 desktop 2411.10323 (2024) code Apache-2.0 Gradio desktop GUI agent for Windows and macOS that needs no Docker, using Claude 3.5 Computer Use over the API or ShowUI and UI-TARS run locally, with multi-display and phone remote control.

Agents · all 49 →

Resource Mentions Environments Paper License Description
Mobile-Agent 9 android, cross-platform 2401.16158 (2024), 2406.01014 (2024), 2508.15144 (2025), 2602.16855 (2026) code MIT Mobile device agent family using visual interface interaction.
Agent S 2 desktop 2410.08164 (2024), 2504.00906 (2025) code Apache-2.0 Simular's open-source framework for computer-use agents that click, type and scroll through desktop and web apps on macOS, Windows and Linux; its third generation, Agent S3, reports 72.6% on OSWorld.
SeeAct 2 web 2401.01614 (2024) — Generalist web agent framework for large multimodal models such as GPT-4V, with a Playwright tool that runs agents on live websites, text-choice and set-of-mark grounding, and a crawler mode.
Agent E 1 web 2407.13032 (2024) code MIT Browser automation agent on the AG2 framework that pairs a planner agent with a browser navigation agent to fill forms, search shops and work through sites, also callable over a local HTTP API.
AppAgent 1 android 2312.13771 (2023) code MIT Multimodal agent that operates Android apps through taps and swipes without back-end access, building a reusable knowledge base from autonomous exploration or from watching human demonstrations.
Auto-GUI / Auto-UI 1 android 2309.11436 (2023) code Apache-2.0 Screenshot-based mobile agent implementation with training and inference code for chain-of-action prediction.

Recently reviewed

Newest editorial approvals of models, datasets, benchmarks, systems, and tools (newest first). New papers appear by date in the topic hubs.

Show the recently reviewed table
Reviewed Resource Kind Environments Description
2026‑11‑06 web-eval-agent tool web MCP server that launches a browser-use agent to run a web app, capture screenshots, console and network logs, and return a UX report to the coding agent.
2026‑11‑06 TestZeus Hercules agent web Open-source testing agent that takes Gherkin features and executes them end to end on web applications through Playwright.
2026‑11‑06 TestDriver SDK framework desktop, web Vitest plugin that spawns ephemeral devices and uses vision-based LLMs to build end-to-end tests of web apps, desktop apps and browser extensions.
2026‑11‑06 Tactile tool desktop, macos Skill, protocol and tool layer that has agents operate software through accessibility semantics first, reading roles, names and states before reaching for screenshots.
2026‑11‑06 SeleniumBase framework web Python browser automation framework for web crawling, end-to-end testing and scraping, with a recorder that generates tests and stealth options.
2026‑11‑06 Qwen Code agent desktop Qwen Code is an open-source coding agent that supports desktop automation through computer-use tools alongside terminal, editor, web, and chat interfaces.
2026‑11‑06 puppeteer-extra framework web A modular plugin framework for Puppeteer, maintained in a monorepo with plugins and related packages.
2026‑11‑06 Playwright MCP Server (ExecuteAutomation) tool web Community MCP server that gives LLMs Playwright browser automation for interacting with web pages, taking screenshots, generating test code and running JavaScript.
2026‑11‑06 pi-computer-use tool desktop, macos, windows, linux Pi extension that gives agents tools to find app windows, observe and search visible controls, click, type, scroll and wait for UI changes in desktop software.
2026‑11‑06 PhantomJS tool web PhantomJS is a scriptable headless WebKit tool for automating web pages, capturing page contents, testing, and monitoring network activity.
2026‑11‑06 OpenManus framework web OpenManus is an open-source agent framework that runs tasks from a terminal and integrates browser execution and screenshots through Browser Use.
2026‑11‑06 mcp-server-browser-use tool web MCP server that wraps browser-use so that an MCP client can automate a real browser, with a CLI, a web dashboard and a deep research mode.

Data files and local queries

Browse the interactive website to search and filter all resources, inspect source evidence, switch between English and Chinese, or download the complete catalog. Website development and publishing.

The English data hub and 中文数据中心 organize public downloads into six groups: complete catalog, RAG documents, citations, topics and comparisons, sources and review status, and schemas. Cloning still gives you the complete source catalog; the hub provides exports from its published snapshot.

The download manifest records each listed export's actual count and unit, scope, byte size, and SHA-256 fingerprint. Compare catalog_sha256 across manifests before combining files from different downloads.

Reviewed research · Computer-use agent guides · Static resource directory · Search and AI discovery

  • Catalog JSONL: one resource per line, with stable IDs, typed fields, provenance, scoped licenses, and review status.
  • Reference graph: outbound references from primary sources.
  • Discovery registry and candidates: source snapshots and where each identifier was found.
  • Review queue: unresolved evidence, unindexed discoveries, and proposals requiring editorial work. Discovered links reviewed as non-resources or as other addresses of indexed resources stay listed with a resolution and their reason.
  • JSON Schema and coverage JSON: machine-checkable contracts and coverage accounting.
  • BibTeX and CSL JSON: reviewed-paper citations with recorded authors, dates, and versions. Each resource page also provides its own record.json.
# Python 3.11+. Search and export require only the standard library.
cd Awesome-CUA
python3 scripts/catalog.py search "grounding" --kind model
python3 scripts/catalog.py search --kind dataset --environment android
python3 scripts/catalog.py search --kind product --status curated --json
python3 scripts/catalog.py get arxiv:2404.07972 --related --json
python3 scripts/catalog.py build
# Creates dist/catalog.sqlite and dist/catalog.json, with no network calls.
python3 scripts/export_rag.py --output dist/rag-documents.jsonl
python3 scripts/export_citations.py --format bibtex --output dist/citations.bib
# Add --include-metadata to include every active record, with its review state intact.

Use SQLite to join resources, environment/topic facets, and reviewed relations. See query examples. data/catalog.jsonl is committed; SQLite and JSON array exports are reproducible local build products.

What “verified” means

State What has been checked
curated AI-assisted editorial review against a first-party source snapshot, with name, scope, summary, and taxonomy
metadata_verified Bibliographic identity, title, authors, and recorded dates checked against retained arXiv or publication sources
candidate A normalized URL or identifier was discovered

A paper, repository, dataset release, model checkpoint, product, and publisher can be separate related resources. Model families and individual checkpoints are explicitly linked where reviewed.

Code, data, weights, and document licenses are separate. Missing values are null.

Maintain and verify

python3 -m venv .venv
.venv/bin/python -m pip install --require-hashes -r requirements-dev.txt
.venv/bin/python scripts/catalog.py validate
.venv/bin/python scripts/catalog.py build --check
.venv/bin/python -m unittest discover -s tests -v
.venv/bin/python scripts/catalog.py build
node --test tests/web.test.mjs
.venv/bin/python scripts/build_site.py

These commands match the checks that Catalog quality runs on every push and pull request, in the same order. They need Python 3.11+ and Node.js 24+, the engines version in package.json (CI uses Python 3.12 and Node.js 24). The web tests use only the built-in Node.js test runner, so nothing needs to be installed with npm. npm run test:web and npm run build:web are shortcuts for the last two commands.

Network discovery and verification are explicit commands; ordinary builds are offline. Changed primary snapshots require another editorial review before publication. Maintenance guide · Methodology · Data dictionary

Acknowledgments and reuse

The source registry credits the community collections and surveys used for discovery. Bibliographic records use retained arXiv, publisher, and author sources; publication observations preserve edition-specific titles, authors, and date precision. Selected assets are checked against publisher repositories, documentation, and model/dataset cards. Upstream READMEs, papers, model weights, and datasets are not redistributed here.

Original repository code and annotations use the MIT license. Linked resources retain their own terms.

To cite the catalog, use CITATION.cff or GitHub's "Cite this repository" button. The data changes with every scan, so pin the exact version you used. Each v* tag has a data release with versioned downloads built from the tagged commit: the catalog as JSONL, JSON and SQLite, the coverage ledger, topic and profile exports, and their SHA256SUMS. Cite the release tag, or record the commit from git rev-parse HEAD in your clone, or link files at that commit, such as https://github.com/reacher-z/Awesome-CUA/blob/<commit>/data/catalog.jsonl. Cutting a release.

排名相邻
同类项目