h-baselines

blank

Contents

• Installation
• Supported Models/Algorithms
  • FuN (FeUdal Networks for Hierarchical Reinforcement Learning)
  • HIRO (Data Efficient Hierarchical Reinforcement Learning)
  • HAC (Learning Multi-Level Hierarchies with Hindsight)
• Environments
  • MuJoCo Environments
  • Mixed Autonomy Traffic
• Citing
• Bibliography
• Useful Links

Installation

To install the h-baselines repository, begin by opening a terminal and set the working directory of the terminal to match

cd path/to/h-baselines

Next, create and activate a conda environment for this repository by running the commands in the script below. Note that this is not required, but highly recommended. If you do not have Anaconda on your device, refer to the provided links to install either Anaconda or Miniconda.

conda env create -f environment.yml
source activate h-baselines

Finally, install the contents of the repository onto your conda environment (or your local python build) by running the following command:

pip install -e .

If you would like to (optionally) validate that the repository was successfully installed and is running, you can do so by executing the unit tests as follows:

nose2

The test should return a message along the lines of:

----------------------------------------------------------------------
Ran XXX tests in YYYs

OK

Supported Models/Algorithms

blank

FuN (FeUdal Networks for Hierarchical Reinforcement Learning)

One of the early works on feudal variants of hierarchical reinforcement learning since the surge of deep neural networks as a viable tool in machine learning, this model attempts to adapt more modern machine learning techniques to the original model presented by [1].

HIRO (Data Efficient Hierarchical Reinforcement Learning)

blank

HAC (Learning Multi-Level Hierarchies with Hindsight)

blank

Environments

This repository contains multiple

MuJoCo Environments

blank

Pendulum blank

Ant Push blank

Ant Fall blank

Mixed Autonomy Traffic

Figure Eight v2 blank

Merge v2 blank

Citing

To cite this repository in publications, use the following:

@misc{h-baselines,
  author = {Kreidieh, Abdul Rahman},
  title = {Hierarchical Baselines},
  year = {2019},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/AboudyKreidieh/h-baselines}},
}

Bibliography

[1] Dayan, Peter, and Geoffrey E. Hinton. "Feudal reinforcement learning." Advances in neural information processing systems. 1993.

[2] Vezhnevets, Alexander Sasha, et al. "Feudal networks for hierarchical reinforcement learning." Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017.

[3] Nachum, Ofir, et al. "Data-efficient hierarchical reinforcement learning." Advances in Neural Information Processing Systems. 2018.

[4] Levy, Andrew, et al. "Learning Multi-Level Hierarchies with Hindsight." (2018).

Useful Links

The following bullet points contain links developed either by developers of this repository or external parties that may be of use to individuals interested in further developing their understanding of hierarchical reinforcement learning:

• https://thegradient.pub/the-promise-of-hierarchical-reinforcement-learning/