GELLO: General, Low-Cost, and Intuitive Teleoperation Framework

GELLO is a general, low-cost, and intuitive teleoperation framework for robot manipulators. This repository contains all the software components for GELLO.

For additional resources:

• Project Website
• Hardware Repository - STL files and build instructions
• ROS 2 Support

Supported Robots

• I2RT YAM
• Franka FR3 (ROS 2 implementation, please refer to the separate documenation in ros2/README.md)
• Franka FER (Panda)
• UR
• xArm
• add your own, see Adding New Robots

Quick Start

git clone https://github.com/wuphilipp/gello_software.git
cd gello_software

Installation

Option 1: Virtual Environment (Recommended)

First, install uv if you don't have it:

curl -LsSf https://astral.sh/uv/install.sh | sh

Create and activate a virtual environment:

uv venv --python 3.11
source .venv/bin/activate  # Run this every time you open a new shell
git submodule init
git submodule update
uv pip install -r requirements.txt
uv pip install -e .
uv pip install -e third_party/DynamixelSDK/python

Option 2: Docker

Install Docker, then:

docker build . -t gello:latest
python scripts/launch.py

ROS 2 Support

Note: GELLO also supports ROS 2 Humble for the Franka FR3 robot. See the ROS 2-specific README in the ros2 directory.

Hardware Configuration

The recommended setup for GELLO is with the I2RT YAM robot arm, using the YAML-based configuration system. This provides the most features and is the best-supported configuration.

Generate YAML Configuration

For the I2RT YAM robot, you can automatically generate your configuration files. This process calibrates the joint offsets and creates configuration files for both simulation and real hardware.

1. Update Motor IDs: Before generating the config, ensure each Dynamixel motor has a unique ID. Install the Dynamixel Wizard and follow these steps:

   1. Connect a single motor to the U2D2 controller.
   2. Open Dynamixel Wizard and scan to detect the motor.
   3. Change the ID to a unique number (e.g., 1 through 7).
   4. Repeat for each motor, ensuring they are in order from base to gripper.

2. Run the Generation Script: With the YAM arm in its default build position (see image below), run the script:

   python scripts/generate_yam_config.py
   

   Follow the prompts in the terminal. This will create configs/yam_auto_generated.yaml for the real robot and configs/yam_auto_generated_sim.yaml for the simulation.

You can now skip to the Usage section.

YAML Configuration System

GELLO uses YAML files in configs/ for configuration. This allows for flexible setup of different robots, environments, and teleoperation parameters. If you have automatically generated your .yaml config files with scripts/generate_yam_config.py, you probably will not need to modify these confings manually.

Sample Configs

Sample configs for the YAM arm and the xarm can be found in configs.

Configuration Components

• Robot Config: Defines robot type, communication parameters, and physical settings.
• Agent Config: Defines GELLO device settings, joint mappings, and calibration.
• DynamixelRobotConfig: Motor-specific settings including IDs, offsets, signs, and gripper.
• Control Parameters: Update rates (hz), step limits (max_steps), and safety settings.

Manual Configuration for Other Robots

Python Configuration for Non-YAM arms

• Most widely supported across different arms
• Located in gello/agents/gello_agent.py
• Uses PORT_CONFIG_MAP dictionary
• Maps USB serial ports to robot configurations

Adding New Robots

To integrate a new robot to the Python configs:

1. Check Compatibility: Ensure your GELLO kinematics match the target robot
2. Implement Robot Interface: Create a new class implementing the Robot protocol from gello/robots/robot.py
3. Add Configuration: Update the configuration system with your robot's parameters

See existing implementations in gello/robots/ for reference:

• panda.py - Franka Panda robot
• ur.py - Universal Robots
• xarm_robot.py - xArm robots
• yam.py - YAM robot

=======

1. Manual gello_agent setup

Set your GELLO and robot arm to a known, matching configuration (see images below) and run the offset detection script.

Command examples:

UR Robot:

python scripts/gello_get_offset.py \
    --start-joints 0 -1.57 1.57 -1.57 -1.57 0 \
    --joint-signs 1 1 -1 1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

Franka FER (Panda):

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 -1.57 0 1.57 0 \
    --joint-signs 1 1 1 1 1 -1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FT7WBG6

I2RT YAM:

python scripts/gello_get_offset.py \
    --start-joints 0 0 0 0 0 0 \
    --joint-signs 1 -1 -1 -1 1 1 \
    --port /dev/serial/by-id/usb-FTDI_USB__-__Serial_Converter_FTAAMLV6-if00-port0

Joint Signs Reference:

• UR: 1 1 -1 1 1 1
• Panda: 1 -1 1 1 1 -1 1
• xArm: 1 1 1 1 1 1 1
• YAM: 1 -1 -1 -1 1 1

Add the generated joint offsets to gello/agents/gello_agent.py in the PORT_CONFIG_MAP.

2. Create Custom YAML Configurations

1. Copy an existing config from configs/ as a template (e.g., yam_passive.yaml).
2. Modify the robot _target_ and parameters for your setup:
   • For hardware: gello.robots.ur.URRobot, gello.robots.panda.PandaRobot, etc.
   • For simulation: gello.robots.sim_robot.MujocoRobotServer
3. Update the agent configuration with your GELLO device settings:
   • port: Your U2D2 device path
   • joint_offsets: From the offset detection script
   • joint_signs: Based on your robot type
   • start_joints: Your GELLO's starting position

Usage

The recommended way to launch GELLO is with a YAML configuration file.

CAN Configuration

Robot arms such as the YAM use a CAN bus to communicate with your machine. If your arm uses a CAN bus, you will need to configure udev rules. First, get your CAN bus ID:

udevadm info -a -p /sys/class/net/can* | grep -i serial

Then open your CAN bus rules using your text editor of choice.

sudo nano /etc/udev/rules.d/90-can.rules

If you only have one arm, add this line:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="", NAME="can_left"

If you have two arms (a bimanual setup), you will need a second line for your right arm. Your bimanual CAN rules file should contain:

SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="", NAME="can_left"
SUBSYSTEM=="net", ACTION=="add", ATTRS{serial}=="", NAME="can_right"

After updating your udev rules, run the following and then unplug and reconnect your CAN devices.

sudo udevadm control --reload-rules && sudo systemctl restart systemd-udevd && sudo udevadm trigger

At this point, your CAN devices are correctly configured. If you encounter CAN connctivity issues after this point run sh scripts/reset_all_can.sh to reset your CAN buses.

YAM GELLO Usage (Recommended)

First, install the YAM-specific dependency:

• YAM: I2RT
• uv pip install -e third_party/i2rt

Testing in Simulation: Launch the simulation with the auto-generated sim config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated_sim.yaml

Real Robot Operation: Launch the real robot with the auto-generated hardware config file:

python experiments/launch_yaml.py --left-config-path configs/yam_auto_generated.yaml

Launching gello_agent for non-YAM arms

For other robots or if not using a YAML configuration, you must launch the robot and controller nodes in separate terminals.

First, install robot-specific dependencies:

• UR: ur_rtde
• Panda: polymetis
• xArm: xArm Python SDK

1. Launch the robot node:

# For simulation
python experiments/launch_nodes.py --robot 

# For real hardware
python experiments/launch_nodes.py --robot 

2. Launch GELLO controller:

python experiments/run_env.py --agent=gello

Troubleshooting

If, when you run generate_yam_config.py, you get an error detecting offsets, you may need to add your user to the dialout user group. To do so, run: sudo usermod -aG dialout $USER And then log out and log back in or restart your computer.s

If some joints in your arm are not behaving as expected, you may need to modify the joint signs of your configuration. Simply invert the affected joint sign(s) in your .yaml or gello_agent.py or physically reverse the installation of the servo.

Optional: Starting Configuration

Use --start-joints to specify GELLO's starting configuration for automatic robot reset:

python experiments/run_env.py --agent=gello --start-joints 

Advanced Features

Data Collection

Collect teleoperation demonstrations with keyboard controls.

For the YAM arm launched with launch_yaml.py, you can append the flag --use-save-interface to enable data saving. This is the recommended method.

python experiments/launch_yaml.py --left-config-path configs/yam_passive.yaml --use-save-interface

After launching, you can begin saving with s and stop saving with q. Data saved will be in the data directory in the root of the project.

For non-YAM setups, use the following:

python experiments/run_env.py --agent=gello --use-save-interface

Process collected data:

python gello/data_utils/demo_to_gdict.py --source-dir=

Bimanual Operation

The recommended way to use bimanual mode is with launch_yaml.py. Pass a config file for the right arm to --right-config-path.

python experiments/launch_yaml.py --left-config-path configs/gello_1.yaml --right-config-path configs/gello_2.yaml

For non-YAM setups, use:

python experiments/launch_nodes.py --robot=bimanual_ur
python experiments/run_env.py --agent=gello --bimanual

FACTR Gravity Compensation

If you want to activate gravity compensation, all the code can be found in gello/factr. It works similarly to the regular launch but for now it's self-contained inside its own subdirectory and supports the YAM arm in sim and in hardware.

The YAML provides important fields that can control the strength of the gravity compensation and friction. Feel free to mess around with the strenght and friction til you attain your desired

One important step is to add the URDF. We have provided the URDF for the active GELLO in the Hardware Repository. You will need to update the path in the YAML to the entry point of the URDF.

python gello/factr/gravity_compensation.py --config configs/yam_gello_factr_hw.yaml

Development

Code Organization

├── scripts/             # Utility scripts
├── experiments/         # Entry points and launch scripts
├── gello/               # Core GELLO package
│   ├── agents/          # Teleoperation agents
│   ├── cameras/         # Camera interfaces
│   ├── data_utils/      # Data processing utilities
│   ├── dm_control_tasks/# MuJoCo environment utilities
│   ├── dynamixel/       # Dynamixel hardware interface
|   ├── factr/           # gravity compensation
│   ├── robots/          # Robot-specific interfaces
│   ├── utils/           # Shared launch and control utilities
│   └── zmq_core/        # ZMQ multiprocessing utilities

Contributing

Install development dependencies and set up pre-commit hooks to ensure code quality before contributing:

uv pip install -r requirements_dev.txt
uv pip install pre-commit
pre-commit install

The codebase uses isort and black for code formatting.

We welcome contributions! Submit pull requests to help make teleoperation more accessible and higher quality.

Citation

@misc{wu2023gello,
    title={GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators},
    author={Philipp Wu and Yide Shentu and Zhongke Yi and Xingyu Lin and Pieter Abbeel},
    year={2023},
}

License & Acknowledgements

This project is licensed under the MIT License (see LICENSE file).

Third-Party Dependencies

• google-deepmind/mujoco_menagerie: Robot models for MuJoCo
• brentyi/tyro: Argument parsing and configuration
• ZMQ: Multiprocessing communication framework

This project uses components from ‘FACTR Teleop: Low-Cost Force-Feedback Teleoperation’ (Apache‑2.0). See https://github.com/RaindragonD/factr_teleop/.