Author: Mike Gimelfarb
This repository supports compilation of RDDL description files into Gurobi's mixed-integer (non-linear) programs, and automated planning tools for optimizing these programs in MDPs.
Note
The Gurobi planners currently determinize all stochastic variables, making it less suitable for highly stochastic problems or problems with (stochastic) dead ends. If you find it is not making sufficient progress on a stochastic problem, or doesn't scale well computationally to your problem, check out the PROST planner (for discrete spaces), the JAX planner (for continuous problems), or the deep reinforcement learning wrappers.
- Installation
- Running the Basic Example
- Running from the Python API
- Customizing Gurobi
- Citing pyRDDLGym-gurobi
The basic requirements are pyRDDLGym>=2.0
and gurobipy>=10.0.1
. To run the basic example, you will also require rddlrepository>=2.0
. Everything can be installed (assuming Anaconda):
# Create a new conda environment
conda create -n gurobiplan python=3.11
conda activate gurobiplan
conda install pip git
# Manually install pyRDDLGym and rddlrepository
pip install git+https://github.com/pyrddlgym-project/pyRDDLGym
pip install git+https://github.com/pyrddlgym-project/rddlrepository
# Install pyRDDLGym-gurobi
pip install git+https://github.com/pyrddlgym-project/pyRDDLGym-gurobi
The basic example provided in pyRDDLGym-gurobi will run the Gurobi planner on a domain and instance of your choosing. To run this, navigate to the install directory of pyRDDLGym-gurobi, and run:
python -m pyRDDLGym_gurobi.examples.run_plan <domain> <instance> <horizon>
where:
<domain>
is the domain identifier as specified in rddlrepository (i.e. Wildfire_MDP_ippc2014), or a path pointing to a validdomain.rddl
file<instance>
instance is the instance identifier (i.e. 1, 2, ... 10) in rddlrepository, or a path pointing to a validinstance.rddl
file<horizon>
is the planning lookahead horizon.
If you are working with the Python API, you can instantiate the environment and planner however you wish:
import pyRDDLGym
from pyRDDLGym_gurobi.core.planner import GurobiStraightLinePlan, GurobiOnlineController
# Create the environment
env = pyRDDLGym.make("domain name", "instance name")
# Create the planner
plan = GurobiStraightLinePlan()
controller = GurobiOnlineController(rddl=env.model, plan=plan, rollout_horizon=5)
# Run the planner
controller.evaluate(env, episodes=1, verbose=True, render=True)
Note, that the GurobiOnlineController
is an instance of pyRDDLGym's BaseAgent
, so the evaluate()
function can be used to streamline interaction with the environment.
The Gurobi compiler and planner run using the Gurobi engine and can be configured by configuring Gurobipy.
Create a gurobi.env
file in the location of your running script, and in it specify the parameters that you would like to pass to Gurobi.
For example, to instruct Gurobi to limit each optimization to 60 seconds, and to print progress during optimization to console:
TimeLimit 60
OutputFlag 1
Parameters can be passed as a dictionary to the model_params
argument of the Gurobi controller:
controller = GurobiOnlineController(rddl=env.model, plan=plan, rollout_horizon=5,
model_params={'NonConvex': 2, 'OutputFlag': 1})
and then the controller can be used as described in the previous section.
The following citation describes the main ideas of the framework. Please cite it if you found it useful:
@inproceedings{gimelfarb2024jaxplan,
title={JaxPlan and GurobiPlan: Optimization Baselines for Replanning in Discrete and Mixed Discrete and Continuous Probabilistic Domains},
author={Michael Gimelfarb and Ayal Taitler and Scott Sanner},
booktitle={34th International Conference on Automated Planning and Scheduling},
year={2024},
url={https://openreview.net/forum?id=7IKtmUpLEH}
}