Installation

The ecosystem targets ROS Noetic on Ubuntu 20.04 with Python 3.8, Gazebo Classic 11, Gymnasium 0.29.1, and Stable Baselines 3 2.x.

Note

ROS Noetic and Gazebo Classic are end-of-life as of 2025. This framework is a research artifact targeting that pinned environment for reproducibility with the published paper. New production work should consider a ROS 2 / modern Gazebo migration — that’s out of scope for the current codebase but tracked as future work.

The fastest installation path is to use the bootstrap script shipped with rl_environments — it installs ROS, the framework core, and the robot drivers in one go. The manual path below is for users who want to understand every step or who already have parts of the stack installed.

Option A — bootstrap script (recommended for a fresh machine)

If you don’t yet have a catkin workspace, the bootstrap script in rl_environments installs the heavy parts in one go:

cd ~
git clone https://github.com/ncbdrck/rl_environments.git
cd rl_environments

chmod +x install_ros_rl.sh
./install_ros_rl.sh            # interactive
# or
./install_ros_rl.sh -n         # non-interactive

What the script installs:

ROS Noetic (if missing)
UniROS (recursively, pulling MultiROS + RealROS as submodules)
The robot drivers — RX200 (Interbotix), Ned2 (Niryo), UR5 (Universal Robots)
hrl-kdl (Noetic-compatible pykdl_utils + hrl_geom)

What the script does not install:

sb3_ros_support — the SB3 wrapper. Needed if you want to train with the convenience layer or use rl_training_validation.
rl_environments itself into the catkin workspace — the script clones it, but you’ll typically want it sitting alongside the other packages.
rl_training_validation — the working training scripts.

After the script finishes, run the following to pick up the rest:

cd ~/catkin_ws/src
git clone -b gymnasium https://github.com/ncbdrck/sb3_ros_support.git
git clone https://github.com/ncbdrck/rl_training_validation.git
# And, if not already there:
git clone https://github.com/ncbdrck/rl_environments.git

cd ~/catkin_ws
rosdep install --from-paths src --ignore-src -r -y
catkin build
source devel/setup.bash

Then jump to Quickstart.

Option B — manual installation

1. ROS Noetic

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt install curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt update
sudo apt install ros-noetic-desktop-full

echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc
source ~/.bashrc

sudo apt install python3-rosdep python3-rosinstall \\
                 python3-rosinstall-generator python3-wstool \\
                 build-essential
sudo rosdep init
rosdep update

2. Catkin tools + workspace

sudo apt-get install python3-catkin-tools

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/
catkin build
echo "source ~/catkin_ws/devel/setup.bash" >> ~/.bashrc
source ~/.bashrc

3. System dependencies

The framework needs xterm (for log windows), MoveIt (motion planning), and PyKDL / kdl_parser_py / trac_ik (kinematics):

sudo apt install \\
    xterm \\
    ros-noetic-moveit \\
    python3-pykdl \\
    ros-noetic-kdl-parser-py \\
    ros-noetic-trac-ik \\
    ros-noetic-urdfdom-py

The Kinematics_pykdl helper relies on a Noetic-compatible fork of pykdl_utils and hrl_geom:

cd ~/catkin_ws/src
git clone https://github.com/ncbdrck/hrl-kdl.git

cd ~/catkin_ws/src/hrl-kdl/pykdl_utils
python3 setup.py build
sudo python3 setup.py install

cd ~/catkin_ws/src/hrl-kdl/hrl_geom
python3 setup.py build
sudo python3 setup.py install

4. Clone the framework

UniROS bundles MultiROS and RealROS as git submodules, so one recursive clone gives you all three packages:

cd ~/catkin_ws/src
git clone --recurse-submodules -b gymnasium https://github.com/ncbdrck/uniros

# Make sure every submodule is on the gymnasium branch
cd ~/catkin_ws/src/uniros
git checkout gymnasium
git submodule update --remote --recursive

cd ~/catkin_ws/src/uniros/multiros
git checkout gymnasium && git pull

cd ~/catkin_ws/src/uniros/realros
git checkout gymnasium && git pull

If you already have MultiROS or RealROS cloned standalone, skip --recurse-submodules and use the existing clones; just be sure they’re on the gymnasium branch.

Workspace layouts

Two layouts are supported, both work:

A. Submodule layout (recommended for new users / RTD) — one recursive clone of UniROS pulls MultiROS and RealROS in as submodules under UniROS/multiros/ and UniROS/realros/.

B. Sibling layout (common for active developers) — clone MultiROS and RealROS as standalone packages alongside UniROS in the catkin src/ directory. This is what an active developer often ends up with: catkin can’t host two packages with the same <name>, so once you start editing MultiROS or RealROS directly it’s easier to have them as siblings of UniROS rather than as nested submodules.

The framework and the docs work the same in both layouts.

Clone the SB3 support package (separate repo, not a submodule):

cd ~/catkin_ws/src
git clone -b gymnasium https://github.com/ncbdrck/sb3_ros_support.git

5. Python dependencies

sudo apt-get install python3-pip

pip3 install -r ~/catkin_ws/src/uniros/uniros/requirements.txt
pip3 install -r ~/catkin_ws/src/uniros/multiros/requirements.txt
pip3 install -r ~/catkin_ws/src/uniros/realros/requirements.txt
pip3 install -r ~/catkin_ws/src/sb3_ros_support/requirements.txt

6. Build the workspace

cd ~/catkin_ws
rosdep install --from-paths src --ignore-src -r -y
catkin build
source devel/setup.bash

7. Verify

python3 -c "import uniros, multiros, realros, sb3_ros_support; print('OK')"

Optional: ready-made envs + training scripts

To get pre-built RX200 / Ned2 / UR5 envs and SB3 training scripts:

cd ~/catkin_ws/src
git clone https://github.com/ncbdrck/rl_environments.git
git clone https://github.com/ncbdrck/rl_training_validation.git

cd ~/catkin_ws
rosdep install --from-paths src --ignore-src -r -y
catkin build
source devel/setup.bash

See Ready-made environments for what’s currently working out of the box, and Training a model for how to train one.

Robot-specific extras

The pre-built envs need the corresponding manufacturer’s ROS packages. Install the ones you’ll use:

Trossen ReactorX-200 (Interbotix arms)

curl 'https://raw.githubusercontent.com/Interbotix/interbotix_ros_manipulators/main/interbotix_ros_xsarms/install/amd64/xsarm_amd64_install.sh' > xsarm_amd64_install.sh
chmod +x xsarm_amd64_install.sh
./xsarm_amd64_install.sh -d noetic -p ~/catkin_ws

Niryo Ned 2

cd ~/catkin_ws/src
git clone https://github.com/NiryoRobotics/ned_ros.git
cd ned_ros
git submodule update --init ros-foxglove-bridge
pip install -r requirements.txt
sudo apt install sqlite3 ffmpeg build-essential -y

Universal Robots UR5

sudo apt install ros-noetic-universal-robots
# or build from source
cd ~/catkin_ws/src
git clone -b noetic-devel https://github.com/ros-industrial/universal_robot.git

Then rebuild the workspace and source it again.

Next step

Continue with Quickstart to launch your first env.