UniROS / MultiROS / RealROS ecosystem

A ROS-based reinforcement-learning framework for robots, spanning Gazebo simulation and real-world hardware. Built on Gymnasium and Stable Baselines 3.

The ecosystem is split into four core framework packages plus two application packages that ship pre-built environments and training scripts. You can use the framework on its own or pull in the applications as ready-made examples.

Core framework: UniROS, MultiROS, RealROS, sb3_ros_support.

Applications: rl_environments, rl_training_validation.

Package	What it provides
UniROS API reference	The unified abstraction layer. Hosts the canonical multiprocessing gym-env proxy class (uniros._proxy.GymProxy) and the shared ROS utility modules used by every package below. Bundles MultiROS and RealROS as git submodules.
MultiROS API reference	Gazebo-based simulation environments. Spawn multiple gym envs in parallel against a single rosmaster, run roscores on arbitrary ports, manage Gazebo physics. An experimental MuJoCo backend is available (see MuJoCo backend (experimental)).
RealROS API reference	The real-hardware counterpart to MultiROS. Same gym API, talks to physical robots instead of Gazebo.
sb3_ros_support API reference	Stable Baselines 3 algorithm wrappers (PPO, A2C, DDPG, TD3, SAC, DQN, plus goal-conditioned variants) configured for ROS-based training scripts.
rl_environments API reference	Pre-built gym environments for the supported robots and tasks.
rl_training_validation API reference	Working training and validation scripts for the pre-built envs.

Get started

• Installation
  • Option A — bootstrap script (recommended for a fresh machine)
  • Option B — manual installation
  • Robot-specific extras
  • Option C — Docker
  • Next step
• Docker
  • When to use Docker
  • What works today
  • Two image variants
  • Quick start
  • Hardware passthrough
  • Active development
  • Roadmap
  • Troubleshooting
  • See also
• Quickstart
  • Verify the workspace
  • A complete first script
  • Press Ctrl+C
  • What’s next
• Ecosystem overview
  • Architecture at a glance
  • Why four packages?
  • Multiprocessing model
  • Lifecycle / cleanup
  • Three supported use cases
  • Framework-agnostic policies
  • Reading further

Environments

• Environments
  • Robots
  • Tasks
  • Variants — every (robot, task) is registered four ways
  • Common safety machinery
  • Reference
• Environment templates
  • The two-layer pattern
  • Template files
  • Recommended workflow
  • Common pitfalls
• Creating a simulation environment
  • 1. Pick a starting point
  • 2. Implement the robot env
  • 3. Implement the task env
  • 4. Register with gymnasium
  • 5. Smoke test
  • Configuration via YAML
  • Common pitfalls
• MuJoCo backend (experimental)
  • How it maps onto the Gazebo design
  • What differs from Gazebo
  • Trying it
  • Status and limitations
  • Differences from the Gazebo backend
• Creating a simulation environment (MuJoCo)
  • 1. Pick a starting point
  • 2. Build the MJCF scene
  • 3. Implement the robot env
  • 4. Implement the task env
  • 5. Register with gymnasium
  • 6. Smoke test
  • 7. Stepping regimes
  • Configuration via YAML
  • Common pitfalls
• Creating a real-hardware environment
  • Differences from sim
  • 1. Pick a starting point
  • 2. Implement the robot env
  • 3. Implement the task env
  • 4. Bring up the robot, then launch the env
  • Sim-to-real transfer
  • Safety checklist

Training

• Training a model
  • Option 1 — Plain Stable Baselines 3
  • Option 2 — sb3_ros_support
  • Option 3 — Any other gymnasium-compatible framework
  • Configuration via YAML (sb3_ros_support)
  • Logging and checkpoints
• Joint sim + real training
  • The idea in one paragraph
  • Algorithm (TD3 / TD3+HER, from the paper)
  • The wrapper that makes this work
  • Process topology
  • Minimal example — TD3 + sim + real
  • Goal-conditioned (TD3 + HER) variant
  • When to use this vs. sim-only or real-only training
  • Tuning notes from the paper
  • Safety checklist
• Using trained models
  • Load and predict
  • Validate sim-trained models on real hardware
  • Deterministic vs stochastic prediction
  • Common pitfalls

API reference

• UniROS API reference
  • Top-level package
  • Gym proxy
  • Shared utilities
• MultiROS API reference
  • Top-level package
  • Gym proxy (re-export)
  • Gazebo lifecycle
  • ROS helpers
  • MoveIt integration
  • Base envs
  • Wrappers
• RealROS API reference
  • Top-level package
  • Gym proxy (re-export)
  • ROS helpers
  • MoveIt integration
  • Base envs
  • Wrappers
• sb3_ros_support API reference
  • Base model
  • Algorithms
  • Goal-conditioned variants (HER)
  • Utilities
• rl_environments API reference
  • Top-level package
  • RX200 — Simulation (Gazebo)
  • RX200 — Real hardware
  • Niryo Ned2 — Simulation
  • Niryo Ned2 — Real hardware
  • Trossen VX300S — Simulation
  • Trossen VX300S — Real hardware
  • Universal Robots UR5e — Simulation
  • Universal Robots UR5e — Real hardware
• rl_training_validation API reference
  • Top-level package
  • Utilities
  • RX200 — Reach
  • Niryo Ned2 — Reach
  • Trossen VX300S — Reach
  • Universal Robots UR5e — Reach
  • Multi-task learning

Development

• Testing
• Contributing
• Known limitations

About these docs

This documentation is written and maintained alongside the UniROS codebase and the Sensors paper. Every change is reviewed before being committed.

Even so, docs can drift out of sync with the code — a renamed API, a stale example, or a subtle factual error. If you find an inaccuracy, please open an issue at github.com/ncbdrck/UniROS/issues.

Citation

If this ecosystem is useful in your work, please cite the paper:

@Article{s25185679,
  AUTHOR  = {Kapukotuwa, Jayasekara and Lee, Brian and Devine, Declan and Qiao, Yuansong},
  TITLE   = {UniROS: ROS-Based Reinforcement Learning Across Simulated and Real-World Robotics},
  JOURNAL = {Sensors},
  VOLUME  = {25},
  YEAR    = {2025},
  NUMBER  = {18},
  PAGES   = {5679},
  URL     = {https://www.mdpi.com/1424-8220/25/18/5679},
  ISSN    = {1424-8220},
  DOI     = {10.3390/s25185679},
}

Indices

• Index

• Module Index

• Search Page