UniROS / MultiROS / RealROS ecosystem

A ROS-based reinforcement-learning framework for robots, spanning Gazebo simulation and real-world hardware. Built on Gymnasium and Stable Baselines 3.

The ecosystem is split into four core framework packages plus two application packages that ship pre-built environments and training scripts. You can use the framework on its own or pull in the applications as ready-made examples.

Core framework: UniROS, MultiROS, RealROS, sb3_ros_support.

Applications: rl_environments, rl_training_validation.

Package	What it provides
UniROS API reference	The unified abstraction layer. Hosts the canonical multiprocessing gym-env proxy class (uniros._proxy.GymProxy) and the shared ROS utility modules used by every package below. Bundles MultiROS and RealROS as git submodules.
MultiROS API reference	Gazebo-based simulation environments. Spawn multiple gym envs in parallel against a single rosmaster, run roscores on arbitrary ports, manage Gazebo physics.
RealROS API reference	The real-hardware counterpart to MultiROS. Same gym API, talks to physical robots instead of Gazebo.
sb3_ros_support API reference	Stable Baselines 3 algorithm wrappers (PPO, A2C, DDPG, TD3, SAC, DQN, plus goal-conditioned variants) configured for ROS-based training scripts.
rl_environments API reference	Pre-built gym environments for the supported robots and tasks.
rl_training_validation API reference	Working training and validation scripts for the pre-built envs.

Get started

• Installation
  • Option A — bootstrap script (recommended for a fresh machine)
  • Option B — manual installation
  • Optional: ready-made envs + training scripts
  • Robot-specific extras
  • Next step
• Quickstart
  • Verify the workspace
  • A complete first script
  • Press Ctrl+C
  • What’s next
• Ecosystem overview
  • Architecture at a glance
  • Why four packages?
  • Multiprocessing model
  • Lifecycle / cleanup
  • Three supported use cases
  • Framework-agnostic policies
  • Reading further

Environments

• Ready-made environments
  • Status legend
  • Robot × task implementation matrix
  • Training script availability
  • Trossen RX200 — simulation, reach
  • Trossen RX200 — simulation, push
  • Trossen RX200 — real hardware, reach
  • Niryo Ned2
  • Universal Robots UR5
• Environment templates
  • The two-layer pattern
  • Template files
  • Recommended workflow
  • Common pitfalls
• Creating a simulation environment
  • 1. Pick a starting point
  • 2. Implement the robot env
  • 3. Implement the task env
  • 4. Register with gymnasium
  • 5. Smoke test
  • Configuration via YAML
  • Common pitfalls
• Creating a real-hardware environment
  • Differences from sim
  • 1. Pick a starting point
  • 2. Implement the robot env
  • 3. Implement the task env
  • 4. Bring up the robot, then launch the env
  • Sim-to-real transfer
  • Safety checklist

Training

• Training a model
  • Option 1 — Plain Stable Baselines 3
  • Option 2 — sb3_ros_support
  • Option 3 — Any other gymnasium-compatible framework
  • Configuration via YAML (sb3_ros_support)
  • Logging and checkpoints
• Joint sim + real training
  • The idea in one paragraph
  • Algorithm (TD3 / TD3+HER, from the paper)
  • The wrapper that makes this work
  • Process topology
  • Minimal example — TD3 + sim + real
  • Goal-conditioned (TD3 + HER) variant
  • When to use this vs. sim-only or real-only training
  • Tuning notes from the paper
  • Safety checklist
• Using trained models
  • Load and predict
  • Validate sim-trained models on real hardware
  • Deterministic vs stochastic prediction
  • Common pitfalls

API reference

• UniROS API reference
  • Top-level package
  • Gym proxy
  • Shared utilities
• MultiROS API reference
  • Top-level package
  • Gym proxy (re-export)
  • Gazebo lifecycle
  • ROS helpers
  • MoveIt integration
  • Base envs
  • Wrappers
• RealROS API reference
  • Top-level package
  • Gym proxy (re-export)
  • ROS helpers
  • MoveIt integration
  • Base envs
  • Wrappers
• sb3_ros_support API reference
  • Base model
  • Algorithms
  • Goal-conditioned variants (HER)
  • Utilities
• rl_environments API reference
  • Top-level package
  • RX200 — Simulation (Gazebo)
  • RX200 — Real hardware
  • Niryo Ned2 — Simulation
  • Niryo Ned2 — Real hardware
• rl_training_validation API reference
  • Top-level package
  • Utilities
  • RX200 reach scripts
  • Multi-task learning

Development

• Testing
• Contributing
• Known limitations

About these docs

This documentation was drafted in collaboration with two AI assistants — Anthropic’s Claude (for writing and structure) and OpenAI’s Codex CLI (for review passes) — working from the UniROS codebase, the Sensors paper, and design decisions made by the author. Each change was reviewed before being committed.

That said, AI-assisted docs can still contain hallucinated APIs, stale examples, or subtle factual errors. If you find an inaccuracy, please open an issue at github.com/ncbdrck/UniROS/issues.

Citation

If this ecosystem is useful in your work, please cite the paper:

@Article{s25185679,
  AUTHOR  = {Kapukotuwa, Jayasekara and Lee, Brian and Devine, Declan and Qiao, Yuansong},
  TITLE   = {UniROS: ROS-Based Reinforcement Learning Across Simulated and Real-World Robotics},
  JOURNAL = {Sensors},
  VOLUME  = {25},
  YEAR    = {2025},
  NUMBER  = {18},
  PAGES   = {5679},
  URL     = {https://www.mdpi.com/1424-8220/25/18/5679},
  ISSN    = {1424-8220},
  DOI     = {10.3390/s25185679},
}

Indices

• Index

• Module Index

• Search Page