Add documentation for continuous action space (#572)

Co-authored-by: Mark Towers <mark.m.towers@gmail.com>

docs/environments.md

@@ -184,6 +184,11 @@ the table above. Together with `mode`, this determines the "flavor" of the game.
 - **full_action_space**: `bool`. If set to `True`, the action space consists of all legal actions on the console. Otherwise, the
 action space will be reduced to a subset.
 
+- **continuous**: `bool`. If set to True, will convert the action space into a Gymnasium [`Box`](https://gymnasium.farama.org/api/spaces/fundamental/#gymnasium.spaces.Box) space.
+Actions passed into the environment are then thresholded to discrete using the `continuous_action_threshold` parameter.
+
+- **continuous_action_threshold**: `float`. This determines the threshold for actions to be thresholded into discrete actions.
+
 - **render_mode**: `str`. Specifies the rendering mode. Its values are:
 	- human: Display the screen and enable game sounds. This will lock emulation to the ROMs specified FPS
 	- rgb_array: Returns the current environment RGB frame of the environment.

docs/gymnasium-interface.md

@@ -1,3 +1,4 @@
+
 # Gymnasium Interface
 
 ALE natively supports [Gymnasium](https://github.com/farama-Foundation/gymnasium). To use these new environments you can simply:
@@ -22,3 +23,22 @@ import gymnasium as gym
 
 env = gym.make('Breakout-v0', render_mode='human')
 ```
+
+## Continuous Action Space
+
+By default, ALE supports discrete actions related to the cardinal directions and fire (e.g., `UP`, `DOWN`, `LEFT`, `FIRE`).
+With `continuous`, Atari environment can be modified to support continuous actions, first proposed in [CALE: Continuous Arcade Learning Environment](https://arxiv.org/pdf/2410.23810).
+
+To initialize an environment with continuous actions, simply use the argument `continuous=True` in the `gymnasium.make`:
+```python
+>>> import gymnasium as gym
+>>> import numpy as np
+>>> import ale_py
+
+>>> gym.register_envs(ale_py)
+>>> env = gym.make("ALE/Breakout-v5", continuous=True)
+>>> env.action_space  # radius, theta and fire where radius and theta for polar coordinates
+Box([0.0, -np.pi, 0.0], [1.0, np.pi, 1.0], np.float32)
+>>> obs, info = env.reset()
+>>> obs, reward, terminated, truncated, info = env.step(np.array([0.9, 0.4, 0.7], dtype=np.float32))
+```