v3.0.0b10

v3.0.0b10 is released

This new version introduces a lot of new features

• Ant walk/run control tasks added and solvable via visual/state based PPO
• New, clean, systematic wrappers/design for fairly evaluating algorithms in simulation for RL, IL etc.
• All end-effector / IK based controllers are now GPU parallelized. Big thank you to @LemonPi for help on batched IK control via their pytorch kinematics library
• four Real2sim environments from SIMPLER have been added, cleaned up, and show good correlation between sim and real success rates and low MMRV relative to original CPU implementations. Evaluation is now 60-100x faster than the real world!
• Trajectory conversion tool for the panda arm is now much more accurate. Tasks like peg insertion / plug charger can convert from pd joint position to any other action space with a much higher success rate.
• Panda wristcam robot had an incorrect initialization on the table scene which has been fixed.
• A base drawing simulation environment with no tasks or demos at the moment
• TwoRobotPickCube-v1 has a new more optimized reward function
• Updated baselines to adhere to new standards of evaluation metrics and reporting
• New sim+rendering benchmarks
• Fix various bugs

Demos of the new drawing environment and real2sim environments below

drawing_special_s_symbol_task.mp4

real2sim_spoon_task.mp4

What's Changed

• [BugFix] Fix bug with processing segmentation images for older torch versions by @StoneT2000 in #532
• [BugFix] Bump sapien dependency in tdmpc2 baseline for mani_skill update by @chenyenru in #527
• Added MS Ant Walk and Run Tasks by @Xander-Hinrichsen in #533
• [Feature] Updated TD-MPC2 evaluation and fixed some bugs by @t-sekai in #538
• Added behavior cloning baselines by @arnavg115 in #531
• [Feature] Align and simplify policy/ML evaluation in ManiSkill by @StoneT2000 in #544
• [Feature] Massively improve success rate of pd joint pos / delta pos conversion to pd ee delta pos/pose controller by @StoneT2000 in #549
• [Feature] Clean up trajectory replay code and more docs on the tool by @StoneT2000 in #552
• panda wristcam initialization added by @jamesahou in #553
• [Feature] Drawing Simulation Base Environment by @StoneT2000 in #555
• [BugFix] Fix CPU articulation net contact forces being wrong shape by @StoneT2000 in #556
• [Feature] Real2Sim Eval Digital Twins by @StoneT2000 in #536
• Solved PPO for TwoRobotPickCube-v1 by @jamesahou in #562
• Behavior cloning bugfixes by @arnavg115 in #565
• [Docs] New benchmarking scripts by @StoneT2000 in #568
• Benchmarking code 2.0 by @StoneT2000 in #573
• [BugFix] Fix panda_v2_gripper.urdf visual mesh suffix (dae->glb) by @kh11kim in #575
• [Docs] More rendering benchmarking by @StoneT2000 in #580
• [Docs] Sim+rendering Benchmarking, matching visuals by @StoneT2000 in #584
• [Docs] Paper upload and update documentation and READMEs by @StoneT2000 in #585
• [Docs] fix typos in paper header by @StoneT2000 in #586
• [Docs] Update docs on controllers and fix video link by @StoneT2000 in #589
• [Feature] Updated TD-MPC2 with CPU mode and new shared metrics by @t-sekai in #578
• [Version] v3.0.0b10 by @StoneT2000 in #591

New Contributors

• @chenyenru made their first contribution in #527
• @jamesahou made their first contribution in #553
• @kh11kim made their first contribution in #575

Full Changelog: v3.0.0b9...v3.0.0b10

v3.0.0b9

What's Changed

• [BugFix] Fix render all mode by @StoneT2000 in #521

Full Changelog: v3.0.0b8...v3.0.0b9

v3.0.0b8

What's Changed

• [Feature] More standardized metrics reporting for diffusion policy by @StoneT2000 in #495
• [Feature] Update PPO RGB baselines by @StoneT2000 in #475
• [Docs] Add missing citations for baselines by @StoneT2000 in #497
• Fix pytorch kinematics bugs by @StoneT2000 in #500
• [Feature] New shader config system and refactors by @StoneT2000 in #499
• [Bug Fix] Add missing _viewer typing by @StoneT2000 in #506
• [BugFix] Support triangle mesh collision shapes in collision mesh generations by @StoneT2000 in #507
• [BugFixes] Fix default robots used for harder table top tasks fix record episode bug when applied to cpu gym wrapped envs by @StoneT2000 in #508
• [Feature] Add floating robotiq_2f_85 gripper support + fix bugs with other robot arms that use that gripper. by @StoneT2000 in #513
• Dm control humanoid ppo learnable stand, walk, run by @Xander-Hinrichsen in #484
• [Docs] Upgrading docs by @StoneT2000 in #516

Full Changelog: v3.0.0b7...v3.0.0b8

v3.0.0b7

Fix a bug with the CPU IK solver and the record episode wrapper.

What's Changed

• [Feature] Diffusion policy baseline by @StoneT2000 in #493

Full Changelog: v3.0.0b6...v3.0.0b7

v3.0.0b6

What's Changed

• use zeros_like in scene setup gpu to avoid nan by @hzaskywalker in #400
• Dense Rews PullCube & LiftPegUpright by @Xander-Hinrichsen in #403
• Update docstring to correctly cite ReplicaCAD as default by @chennisden in #410
• [BugFix] Replaces the default gymnasium timelimit wrapper with a batched/torch version when gym.make is used by @StoneT2000 in #416
• [Feature] Expose articulation acceleration values by @StoneT2000 in #415
• [BugFix] Fix bug where motion planning example code kept trying to recreate grasp pose visual objects by @StoneT2000 in #428
• [Docs] In ReplicaCAD scene builder, fix comment documenting ASSET_DIR location by @chennisden in #430
• Remove unnecessary global declaration by @chennisden in #433
• [Feature] add G1 robot by @matheecs in #405
• [Feature] Add a floating panda gripper and update missing docs on new robots by @StoneT2000 in #436
• Initialize qvel when resetting for control_mode = "pd_joint_pos_vel" by @sean1295 in #432
• [Feature] Add RLPD Baseline by @StoneT2000 in #437
• Update Docs by @Xander-Hinrichsen in #441
• [Feature] Support ppo_rgb with rgb only, no state. And some bug fixes for ppo_rgb for locomotion tasks by @StoneT2000 in #446
• [BugFix] Fix bug where linear/angular velocities are swapped in GPU sim vs CPU sim by @StoneT2000 in #452
• [Docs] Update RFCL baseline documentation to pin compatible jax/torch… by @StoneT2000 in #453
• Control Hopper Env, Stand + Hop by @Xander-Hinrichsen in #440
• [Feature] New assets for floor/lighting and support for recording all envs in one scene in the parallel GUI render mode by @StoneT2000 in #445
• [Feature] SIm/Render device selection by @StoneT2000 in #455
• [Feature] Automatic asset download checks by @StoneT2000 in #456
• simbackend for cpu by @hzaskywalker in #459
• [Feature] New assets, vision based quadruped reach example, and new render all mode by @StoneT2000 in #461
• [Feature] Add unitree go 2 task with simplified collisions and new penalty for quadruped reach for better gaits by @StoneT2000 in #463
• [Feature] Quadruped spin control task by @StoneT2000 in #464
• [Docs] Update renders with new floor texture and update docs on humanoid tasks by @StoneT2000 in #465
• [BugFix] Fix memory leak by forcing a gc.collect upon reconfiguring by @StoneT2000 in #467
• [BugFix] Fix multi-gpu setup documentation and future proof some code by @StoneT2000 in #468
• [Docs] Documentation on proper RL benchmarking and state based PPO results by @StoneT2000 in #470
• [Feature] Basic support for stable baselines 3 by @StoneT2000 in #476
• [Feature] Code to support future render system by @StoneT2000 in #480
• [BugFix] Add back pytorch kinematics which is likely more stable by @StoneT2000 in #485
• [Feature] Add in vectorized TDMPC-2 Baseline by @t-sekai in #451
• [Docs] Update docs and fix some bugs with autodoc by @StoneT2000 in #487
• [Feature] Align RFCL and RLPD training metrics with other RL baselines by @StoneT2000 in #489
• [BugFix] Adding missing init.py file for users who pip install from git by @StoneT2000 in #491

New Contributors

• @chennisden made their first contribution in #410
• @matheecs made their first contribution in #405
• @sean1295 made their first contribution in #432
• @t-sekai made their first contribution in #451

Full Changelog: v3.0.0b5...v3.0.0b6

v3.0.0b5

What's Changed

• [Feature] SAC Parallelized Baseline by @StoneT2000 in #345
• [Feature] Add Panda Wrist Cam option to PickSingleYCB task by @StoneT2000 in #349
• [Feature] Improved QuadrupedReach rewards and make env more markov for faster training by @StoneT2000 in #350
• [Feature] Add back the high quality AI2THOR Scenes by @StoneT2000 in #352
• [Docs] New documentation on scene datasets by @StoneT2000 in #353
• [Docs] Old hyperlink fix by @CreativeNick in #356
• [Feature] New CPU/GPU contact forces/impulses unified API by @StoneT2000 in #354
• [Docs] Fix docs on contacts API by @StoneT2000 in #359
• [BugFix]: Remove extraneous print() by @arth-shukla in #360
• [Fix]: Fix Fetch grip directions by @arth-shukla in #361
• [Feature] SAC + reverse forward curriculum learning by @StoneT2000 in #365
• [Feature] Parallel Rendering Benchmark Code by @StoneT2000 in #363
• [Docs] Some benchmark results on 4090 by @StoneT2000 in #368
• [BugFix] Fix trajectory replay saving the wrong first observation if using use first env state by @StoneT2000 in #369
• [Feature] Faster jax RFCL by @StoneT2000 in #373
• [Docs] Fix docs links for RFCL by @StoneT2000 in #374
• [Feature] Add more scripts for generating motionplanning demos, update demo links to new datasets on HF by @StoneT2000 in #372
• [BugFix] allow set collision group bit to 0 in set_collision_group_bit by @arth-shukla in #376
• [CSE 276F Submission] PokeCube-v1 by @xixinzhang in #375
• Added RollBall env by @guru-narayana in #366
• [Docs] Fix docs on roll ball and update the demo video by @StoneT2000 in #380
• fix data set & apply_gpu order in setup_gpu by @hzaskywalker in #381
• [CSE 276F Submission] PlaceSphere-v1 by @astonastonaston in #379
• [CSE 276F Submission] PushT-v1 by @Xander-Hinrichsen in #378
• [Feature] Parallel render viewer by @StoneT2000 in #383
• [Feature] - Disable some collisions on fetch to make it run faster by @StoneT2000 in #385
• fetch move forward/backward only by @arth-shukla in #386
• [BugFix] Fix PD joint position to PD ee delta pose control conversion by @StoneT2000 in #388
• [Docs] Update ppo examples by @StoneT2000 in #389
• [BugFix] Fix bug with recorded motion planning demos of some tasks. by @StoneT2000 in #391

New Contributors

• @CreativeNick made their first contribution in #356
• @xixinzhang made their first contribution in #375
• @guru-narayana made their first contribution in #366
• @hzaskywalker made their first contribution in #381
• @astonastonaston made their first contribution in #379
• @Xander-Hinrichsen made their first contribution in #378

Full Changelog: v3.0.0b4...v3.0.0b5

What's Changed

• [Feature] SAC Parallelized Baseline by @StoneT2000 in #345
• [Feature] Add Panda Wrist Cam option to PickSingleYCB task by @StoneT2000 in #349
• [Feature] Improved QuadrupedReach rewards and make env more markov for faster training by @StoneT2000 in #350
• [Feature] Add back the high quality AI2THOR Scenes by @StoneT2000 in #352
• [Docs] New documentation on scene datasets by @StoneT2000 in #353
• [Docs] Old hyperlink fix by @CreativeNick in #356
• [Feature] New CPU/GPU contact forces/impulses unified API by @StoneT2000 in #354
• [Docs] Fix docs on contacts API by @StoneT2000 in #359
• [BugFix]: Remove extraneous print() by @arth-shukla in #360
• [Fix]: Fix Fetch grip directions by @arth-shukla in #361
• [Feature] SAC + reverse forward curriculum learning by @StoneT2000 in #365
• [Feature] Parallel Rendering Benchmark Code by @StoneT2000 in #363
• [Docs] Some benchmark results on 4090 by @StoneT2000 in #368
• [BugFix] Fix trajectory replay saving the wrong first observation if using use first env state by @StoneT2000 in #369
• [Feature] Faster jax RFCL by @StoneT2000 in #373
• [Docs] Fix docs links for RFCL by @StoneT2000 in #374
• [Feature] Add more scripts for generating motionplanning demos, update demo links to new datasets on HF by @StoneT2000 in #372
• [BugFix] allow set collision group bit to 0 in set_collision_group_bit by @arth-shukla in #376
• [CSE 276F Submission] PokeCube-v1 by @xixinzhang in #375
• Added RollBall env by @guru-narayana in #366
• [Docs] Fix docs on roll ball and update the demo video by @StoneT2000 in #380
• fix data set & apply_gpu order in setup_gpu by @hzaskywalker in #381
• [CSE 276F Submission] PlaceSphere-v1 by @astonastonaston in #379
• [CSE 276F Submission] PushT-v1 by @Xander-Hinrichsen in #378
• [Feature] Parallel render viewer by @StoneT2000 in #383
• [Feature] - Disable some collisions on fetch to make it run faster by @StoneT2000 in #385
• fetch move forward/backward only by @arth-shukla in #386
• [BugFix] Fix PD joint position to PD ee delta pose control conversion by @StoneT2000 in #388
• [Docs] Update ppo examples by @StoneT2000 in #389
• [BugFix] Fix bug with recorded motion planning demos of some tasks. by @StoneT2000 in #391

New Contributors

• @CreativeNick made their first contribution in #356
• @xixinzhang made their first contribution in #375
• @guru-narayana made their first contribution in #366
• @hzaskywalker made their first contribution in #381
• @astonastonaston made their first contribution in #379
• @Xander-Hinrichsen made their first contribution in #378

Full Changelog: v3.0.0b4...v3.0.0b5

v3.0.0b4

v3.0.0b4 Release

Major changes:

• The open cv (cv2) package is now no longer a dependency, making it easier for ManiSkill to run on more systems.
• EE based control (PDEEPoseController) has been overhauled and documentation has been updated to explain it much more clearly. Importantly there are now 4 frames of control (arising from the product of 2 rotation and 2 translation frames) and the default has been changed to be the more intuitive default (e.g. +Z is actually upwards / in the root link frame). Moreover the CPU sim default frame is the same frame used by the GPU sim. See https://maniskill.readthedocs.io/en/latest/user_guide/concepts/controllers.html#pd-ee-end-effector-pose. This also means previous policies trained on ee controllers (like pd_ee_delta_pose) will not work the same as before and do not directly transfer to this version.
• SAPIEN dependency has been upgraded, which fixes a number of bugs particularly around support for older / less common GPUs

What's Changed

• [BugFix] Windows/wsl bug fixes by @StoneT2000 in #333
• [Feature] Remove cv2 by @StoneT2000 in #335
• [Feature] Update scene builder api by @arth-shukla in #315
• [BugFix] Fix bug where Actors/Root Links did not accept raw tensors for poses by @StoneT2000 in #336
• [BugFix] Fix typo in ppo baselines with printed evaluation step values by @StoneT2000 in #337
• [BugFix] Update motionplanner.py by @PartyPenguin in #338
• [Feature] Update and document ee-control controllers by @StoneT2000 in #339
• [Docs] Fix video links by @StoneT2000 in #340
• [Feature] Support any task loading any robot and print warning instead of assert by @StoneT2000 in #341

New Contributors

• @PartyPenguin made their first contribution in #338

Full Changelog: v3.0.0b3...v3.0.0b4

v3.0.0b3

What's Changed

• [Docs] Fix docs for visualizing reset distributions demo by @StoneT2000 in #313
• [Docs] Fix typo in readmes/docs by @StoneT2000 in #314
• [BugFix] Fix fast-kinematics dependency by @StoneT2000 in #318
• [BugFix] fix bug for demo robot code where mimic joints could not work with keyframe actions by @StoneT2000 in #319
• [BugFix] copy info dict to avoid self reference memory leak in the gymnasium vector env wrapper by @StoneT2000 in #320
• [BugFix] Fix bug with ppo example code under new wrapper API fix by @StoneT2000 in #321
• version bump by @StoneT2000 in #322

Full Changelog: v3.0.0.b2...v3.0.0b3

v3.0.0.b2

ManiSkill 3 Beta Release

Many many changes compared to ManiSkill 2. ManiSkill v3 is now a GPU accelerated robotics simulator, with extremely fast parallel rendering supporting all kinds of robot learning workflows. There are a lot of new robots/tasks and features, find out more on our documentation: https://maniskill.readthedocs.io/en/latest/

A short summary of some key features:

• GPU parallelized visual data collection system. On the high end you can collect RGBD + Segmentation data at 20k FPS with a 4090 GPU, 10-100x faster compared to most other simulators.
• Example tasks covering a wide range of different robot embodiments (quadruped, mobile manipulators, single-arm robots) as well as a wide range of different tasks (table-top, locomotion, dextrous manipulation)
• GPU parallelized tasks, enabling incredibly fast synthetic data collection in simulation
• GPU parallelized tasks support simulating diverse scenes where every parallel environment has a completely different scene/set of objects
• Flexible task building API that abstracts away much of the complex GPU memory management code

What's Changed

• Setup Automatic Pypi uploads on release by @StoneT2000 in #137
• Update README.md by @StoneT2000 in #280
• Initial ManiSkill 3 merge into main branch by @StoneT2000 in #287
• Improve rgb ppo, update code for latest sapien release by @StoneT2000 in #291
• Update quadruped envs and procedural generated floor texture by @StoneT2000 in #279
• Improved demo code, moving some robot assets to other repos, bug fixes by @StoneT2000 in #292
• Unitree go2 initial work by @StoneT2000 in #293
• Quadruped rl by @StoneT2000 in #294
• Cartpole by @StoneT2000 in #296
• Updated robot tutorial by @StoneT2000 in #297
• More docs, robots by @StoneT2000 in #298
• code refactoring for some sensors, base env is always batched+torch unless user uses wrapper, make some code cleaner by @StoneT2000 in #299
• Allow import mani_skill to get access to envs + bug fix with pick clutter env by @StoneT2000 in #301
• update custom robot docs, property to check if gpu sim is enabled, code to test gpu sim of robot by @StoneT2000 in #302
• More docs on existing tasks by @StoneT2000 in #303
• Unify the way non primitive actors can be built by @StoneT2000 in #304
• Refactoring ._scene->.scene, refactor custom task docs, more custom task docs, doc on simulation 101, some bug fixes for a missing function by @StoneT2000 in #305
• More docs on demo scripts, fix old demo scripts, add some missing videos of tasks by @StoneT2000 in #306
• Plug charger task by @StoneT2000 in #307
• version bump + update README + prepare for beta release by @StoneT2000 in #308

Full Changelog: v0.5.3...v3.0.0.b1

v3.0.0.dev13

What's Changed

• Fix camera not accepting intrinsic argument, small refactor of add_camera API, and add peg insertion to test matrices by @StoneT2000 in https://github.com/haosulab/ManiSkill2/pull/267
• Add segmentation data w/ docs + demos, fix replay trajectory test, change OrderedDict to dict in some places by @StoneT2000 in https://github.com/haosulab/ManiSkill2/pull/270

Full Changelog: haosulab/ManiSkill2@v3.0.0.dev12...v3.0.0.dev13