Vectorization of environment using cpu in windows

[j-thapa]

Hello,

I am trying to train RL models using vectorized environments on CPU in windows, is it feasible or not; while I am trying to do so am getting RuntimeError: Some environments have an observation space different.

I am using custom environment but even when using existing environment like PushCube am getting the same error.

For reference here is the section of code where am trying to create the vectorized environments.

Define the environment

env_id = "PickCube-v1" # Your chosen environment
obs_mode = "rgbd" # Observation mode
control_mode = "pd_ee_delta_pos" # Control mode
reward_mode = "normalized_dense" # Reward mode
robot_uids = "panda" # Robot type

def cpu_make_env(env_id, env_kwargs=dict()):
def thunk():
env = gym.make(env_id,
obs_mode=obs_mode,
reward_mode=reward_mode,
control_mode=control_mode,
robot_uids=robot_uids)
env = CPUGymWrapper(env)
return env
return thunk

if __name__ == "__main__":
import multiprocessing as mp
mp.freeze_support()  # Necessary for Windows

num_eval_envs = 8
env_kwargs = dict(obs_mode="rgbd")  # Modify your env_kwargs here

vector_cls = gym.vector.SyncVectorEnv if num_eval_envs == 1 else lambda x: gym.vector.AsyncVectorEnv(x, context="spawn")

eval_envs = vector_cls([cpu_make_env(env_id, env_kwargs) for _ in range(num_eval_envs)])

# Evaluation loop
obs, _ = eval_envs.reset(seed=0)

for _ in range(400):
    action = eval_envs.action_space.sample()  # Replace with your policy action
    obs, rew, terminated, truncated, info = eval_envs.step(action)
    print(rew)

I followed and made some changes as required from the example shown in https://maniskill.readthedocs.io/en/latest/user_guide/reinforcement_learning/setup.html#evaluation

Thanks