Skip to content

Installation

Prerequisites

Simply run the following command for a quick start

git clone https://github.com/vwxyzjn/cleanrl.git && cd cleanrl
poetry install

poetry install hangs / stucks

Since 1.2+ poetry added some keyring authentication mechanisms that may cause poetry install hang or stuck. See python-poetry/poetry#1917. To fix this issue, try

export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
poetry install

Working with different CUDA versions for torch

By default, the torch wheel is built with CUDA 10.2. If you are using newer NVIDIA GPUs (e.g., 3060 TI), you may need to specifically install CUDA 11.3 wheels by overriding the torch dependency with pip:

poetry run pip install "torch==1.12.1" --upgrade --extra-index-url https://download.pytorch.org/whl/cu113

Working with PyPI mirrors

Users in some countries (e.g., China) can usually speed up package installation via faster PyPI mirrors. If this helps you, try appending the following lines to the pyproject.toml at the root of this repository and run poetry install

[[tool.poetry.source]]
name = "douban"
url = "https://pypi.doubanio.com/simple/"
default = true

Install via pip

While we recommend using poetry to manage environments and dependencies, the traditional requirements.txt are available:

# core dependencies
pip install -r requirements/requirements.txt

# optional dependencies
pip install -r requirements/requirements-atari.txt
pip install -r requirements/requirements-mujoco.txt
pip install -r requirements/requirements-mujoco_py.txt
pip install -r requirements/requirements-procgen.txt
pip install -r requirements/requirements-envpool.txt
pip install -r requirements/requirements-pettingzoo.txt
pip install -r requirements/requirements-jax.txt
pip install -r requirements/requirements-docs.txt
pip install -r requirements/requirements-cloud.txt

Optional Dependencies

CleanRL makes it easy to install optional dependencies for common RL environments and various development utilities. These optional dependencies are defined at the pyproject.toml as poetry dependency groups:

[tool.poetry.group.atari]
optional = true
[tool.poetry.group.atari.dependencies]
ale-py = "0.7.4"
AutoROM = {extras = ["accept-rom-license"], version = "^0.4.2"}
opencv-python = "^4.6.0.66"

[tool.poetry.group.procgen]
optional = true
[tool.poetry.group.procgen.dependencies]
procgen = "^0.10.7"

You can install them using the following command

poetry install -E atari
poetry install -E mujoco
poetry install -E mujoco_py
poetry install -E dm_control
poetry install -E procgen
poetry install -E envpool
poetry install -E pettingzoo
poetry install -E jax
poetry install -E optuna
poetry install -E docs
poetry install -E cloud