JarvisHub

JarvisHub

An Open Harness for Canvas-Native Multimodal Creative Agents

Turn an editable canvas into shared project state for long-horizon creative work.

📮 Updates

• [2026.7.28] Repository open-sourced and paper uploaded.

🎬 Demo Video

Click the image above to watch the demo video.

Overview

JarvisHub is a canvas-native creative agent harness for long-horizon multimodal creation. The canvas is not only a visual interface. It is the workspace shared by people and agents, external memory that the agent can inspect, a protocol-constrained action space, and persistent state for artifacts, dependencies, versions, status, and feedback.

Prompt tools, chatbot agents, and node workflows each cover part of creative production. JarvisHub keeps the complete project state visible and editable on one canvas.

What We're Building (And Why It's Different)

Existing creative systems leave important gaps:

• Prompt-to-output tools are effective at producing individual assets, but usually hide intermediate decisions, failed trials, and revision history.
• Chatbot agents can invoke tools and follow multi-step instructions, but linear conversation is a poor representation for spatial layouts, asset dependencies, version branches, and local edit targets.
• Node-based workflows expose execution steps, but commonly rely on manually specified pipelines rather than a project state an agent can continuously inspect, repair, and extend.

JarvisHub makes the canvas the source of truth. Text, references, character sheets, scenes, storyboards, images, videos, audio, webpage previews, presentation slides, candidates, and revision notes become addressable nodes and links. On every turn, the agent observes the current canvas, selects a permitted action, invokes a model or tool, and commits the returned artifacts and evidence to the same workspace.

The goal is not to replace generation models. JarvisHub provides an open and inspectable runtime for building agents that preserve context, orchestrate tools, use feedback, and recover from failure across extended creative workflows.

✨ What You Can Do

The JarvisHub paper demonstrates three representative long-horizon tasks:

JarvisHub also exposes the reusable state behind the final deliverable:

• Keep prompts, references, drafts, candidates, versions, and feedback on the canvas.
• Express references, generation dependencies, version lineage, and workflow continuation with typed nodes and edges.
• Invoke image, video, audio, code, browser, file, document, presentation, and MCP-backed tools.
• Use Skills for reusable procedures and Memory for preferences and prior decisions.
• Use Subagents to explore independent subtasks before the parent agent integrates useful results.
• Record requests, actions, observations, feedback, repair decisions, and canvas updates for inspection and recovery.

🧠 How JarvisHub Works

JarvisHub is organized into three core layers:

🚀 Quick Start

Requirements

• Node.js v24.15.0
• pnpm
• Docker and Docker Compose for the default local PostgreSQL database
• Python 3.10+ with python-pptx and Pillow

One-command startup

./run.sh

On the first run, the launcher automatically:

• installs missing workspace dependencies;
• starts and waits for local PostgreSQL;
• validates the vendored PPT Master runtime;
• selects a compatible Python executable;
• builds the Agents CLI;
• starts Web, API, Agents Bridge, and Trace Viewer.

The launcher stays in the foreground and aggregates service logs. Press Ctrl+C to stop the processes started by that run.

Docker Compose startup

./scripts/dev.sh docker --build

Docker mode starts Web, API, Agents Bridge, Trace Viewer, PostgreSQL, and Redis. The first run pulls images and installs dependencies, so it can take several minutes; dependencies are cached in Docker volumes and later starts are faster.

If Docker Hub pulls are slow or unavailable, use a registry mirror for that run:

DOCKERHUB_REGISTRY=mirror.gcr.io/library ./scripts/dev.sh docker --build

Default Docker endpoints:

Override host ports when the defaults are already in use:

WEB_PORT=5174 API_PORT=18788 AGENTS_PORT=18799 TRACE_API_PORT=15781 TRACE_WEB_PORT=15782 POSTGRES_PORT=15432 ./scripts/dev.sh docker --build

Stop the Docker services:

docker compose -f apps/hono-api/docker-compose.yml down

Common commands

./run.sh             # Full startup
./run.sh restart     # Rebuild Agents and restart every service
./run.sh stop        # Stop local services

Maintenance commands

./run.sh --install   # Explicitly rerun pnpm install
./run.sh --no-build  # Skip the Agents build
./run.sh --clean     # Clear Agents caches before building
./run.sh db          # Start PostgreSQL only
./run.sh db:logs     # Follow PostgreSQL logs
./run.sh db:stop     # Stop PostgreSQL

Service endpoints

⚙️ Configuration

run.sh is the single entry point for local development. Override defaults directly from the shell:

WEB_PORT=5176 API_PORT=8789 ./run.sh
NPM_REGISTRY=https://registry.npmjs.org ./run.sh
LOCAL_PROXY_URL=http://127.0.0.1:7890 ./run.sh
DATABASE_URL=postgresql://user:password@host:5432/database ./run.sh
PPT_MASTER_PYTHON=/absolute/path/to/python3 ./run.sh

Model providers, API keys, and default models are configured in the JarvisHub Web UI and stored in the database. The application can start without model configuration, but generation tasks that depend on those models will not run.

Shared test R2

By default, ./run.sh connects to a public and disposable shared test R2 bucket so open-source users can quickly try asset upload and generated-media hosting.

Data warning: every user shares the same test space, and objects may be accessed, overwritten, cleaned up, or deleted by others. Do not upload private data, business materials, credentials, or anything that must be retained.

Stable deployments and private data must use a separate R2 bucket:

R2_ACCESS_KEY_ID=your-access-key \
R2_SECRET_ACCESS_KEY=your-secret-key \
R2_BUCKET_URL=https://account-id.r2.cloudflarestorage.com/your-bucket \
R2_REGION=auto \
R2_PUBLIC_BASE_URL=https://assets.example.com \
./run.sh

Production deployments should inject database, storage, and infrastructure credentials through the deployment platform. Never commit private credentials to the repository.

🗂 Repository Layout

apps/
  web/              # Editable canvas and frontend application
  hono-api/         # API, project state, Protocol Bridge, and storage integration
  agents-cli/       # Agent Runtime, Skills, Memory, Tools, and collaboration
packages/
  canvas-layout/    # Canvas layout capabilities
  schemas/          # Shared data and protocol schemas
tools/
  trace-viewer/     # Agent trajectory viewer
vendor/
  ppt-master/       # Vendored PPT Master runtime
run.sh              # Single local-development entry point

🔬 Research Scope and Limitations

• The current paper reports qualitative demonstrations rather than a completed benchmark or leaderboard.
• JarvisHub focuses on orchestration, project state, and traceability. Final artifact quality still depends on external models and tools.
• The Protocol Bridge makes actions explicit, inspectable, and recoverable, but cannot guarantee that every creative decision is semantically correct.
• Trajectories may support analysis and future training, but require quality filtering, user consent, anonymization, and copyright review.
• The open-source runtime is evolving, and interfaces or data structures may change.

📖 Citation

@article{lin2026jarvishub,
  title={JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents},
  author={Lin, Yunlong and Lin, Zixu and Xing, Zhaohu and Li, Biqiang and Li, Chenxin and Wang, Haonan and Wu, Haitao and Liu, Hengyu and Chen, Jianghai and Feng, Kaituo and others},
  journal={arXiv preprint arXiv:2607.23588},
  year={2026}
}

🙏 Acknowledgements

JarvisHub builds on the open-source foundation of TapCanvas and integrates ppt-master for presentation-generation workflows. We are grateful to the maintainers and contributors of both projects for making their work openly available.

License

JarvisHub is licensed under the Apache License 2.0.

If JarvisHub helps your work, consider giving the project a ⭐ Star.