-
Notifications
You must be signed in to change notification settings - Fork 647
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request] use one pass to compute mean and variance of recorded data #452
Comments
@tanjunyao7 Yes! it's on our todo list but we don't have the bandwidth as of now. If you have time could you please create a PR? That would be extremely helpful!!! cc @michel-aractingi for visibility |
yes, I could create a PR. I'll close this issue. |
sorry I decided to paste the code here since I don't have time to write the test script. It's manually tested by computing the original result and the new result of the same data. Here is the code snippet:
|
Hi,
first of all, thanks for the great work.
I recorded 50 episodes with a real robot with each episode lasting 20 seconds. When the recording is finished, the statistics of the data is computed for the normalization. However, the computation costs almost one hour. After investigating the code, I found that it iterates the data twice, first for the computation of mean, second for variance.
lerobot/lerobot/common/datasets/compute_stats.py
Lines 102 to 149 in 9257348
I believe both the mean and variance can be computed in a single pass, halving the total computation time. Are there any plan for this improvement?
The text was updated successfully, but these errors were encountered: