feat: enable to use multiple rgb encoders per camera in diffusion policy #484
+29
−10
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this does
This PR enables to configure multiple rgb encoders for each cameras, which is done in the original diffusion policy paper. Not for sure in general setting, but at least in my task, this seems to increase the success rate of deployment slightly.
Current diffusion policy shared the single RGB encoder for all cameras.
This fixes #483
How it was tested
a) regression test
Prepare the following script and check that the behavior of the policy does not change before and the after the PR by comparing the hash value of the output.
Before this PR
After this PR
You can see the match of hash values.
b) validity check
The above test include the case (see case3 ), which is run only when the
DiffusionConfig
hasrgb_encoder_per_camera
attribute, so activate only after this PR. And as you can see, there is no error occured and output the action.c) tested on real robot
I trained the diffusion policy with the
rgb_encoder_per_camera=True
, and checked that policy worked well. I cannot say concrete stuff but the performance is slightly better than the original.How to checkout & try? (for the reviewer)
Try the above script and please check the hash value before and after this PR. Before the run, you need to save the following yaml file as
'stats-diffusion.yaml
.