diff --git a/docs/environments.md b/docs/environments.md index 34f3986e..42964831 100644 --- a/docs/environments.md +++ b/docs/environments.md @@ -184,6 +184,11 @@ the table above. Together with `mode`, this determines the "flavor" of the game. - **full_action_space**: `bool`. If set to `True`, the action space consists of all legal actions on the console. Otherwise, the action space will be reduced to a subset. +- **continuous**: `bool`. If set to True, will convert the action space into a Gymnasium [`Box`](https://gymnasium.farama.org/api/spaces/fundamental/#gymnasium.spaces.Box) space. +Actions passed into the environment are then thresholded to discrete using the `continuous_action_threshold` parameter. + +- **continuous_action_threshold**: `float`. This determines the threshold for actions to be thresholded into discrete actions. + - **render_mode**: `str`. Specifies the rendering mode. Its values are: - human: Display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS - rgb_array: Returns the current environment RGB frame of the environment. diff --git a/docs/gymnasium-interface.md b/docs/gymnasium-interface.md index 1547b811..a43d4187 100644 --- a/docs/gymnasium-interface.md +++ b/docs/gymnasium-interface.md @@ -1,3 +1,4 @@ + # Gymnasium Interface ALE natively supports [Gymnasium](https://github.com/farama-Foundation/gymnasium). To use these new environments you can simply: @@ -22,3 +23,22 @@ import gymnasium as gym env = gym.make('Breakout-v0', render_mode='human') ``` + +## Continuous Action Space + +By default, ALE supports discrete actions related to the cardinal directions and fire (e.g., `UP`, `DOWN`, `LEFT`, `FIRE`). +With `continuous`, Atari environment can be modified to support continuous actions, first proposed in [CALE: Continuous Arcade Learning Environment](https://arxiv.org/pdf/2410.23810). + +To initialize an environment with continuous actions, simply use the argument `continuous=True` in the `gymnasium.make`: +```python +>>> import gymnasium as gym +>>> import numpy as np +>>> import ale_py + +>>> gym.register_envs(ale_py) +>>> env = gym.make("ALE/Breakout-v5", continuous=True) +>>> env.action_space # radius, theta and fire where radius and theta for polar coordinates +Box([0.0, -np.pi, 0.0], [1.0, np.pi, 1.0], np.float32) +>>> obs, info = env.reset() +>>> obs, reward, terminated, truncated, info = env.step(np.array([0.9, 0.4, 0.7], dtype=np.float32)) +```