Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

set temperature to 0 to make results reproducible #77

Merged
merged 2 commits into from
Oct 11, 2023

Conversation

30stomercury
Copy link
Contributor

Set temperature to 0 for reproducible asr results. See https://github.com/fairinternal/seamless_common/pull/289 for the explanation. @uralik we might also need to fix this line?

Note that this is a temporal workaround for current SimulEval as we may unify the evaluation pipeline in the near future.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 20, 2023
@uralik
Copy link

uralik commented Sep 20, 2023

@uralik we might also need to fix this line?

for translation use case non-determinism was more desired compared to degeneration (repetitions etc. which might happen when whisper does not fall back to sampling). thats why I didnt change that part

@30stomercury 30stomercury merged commit 411a73d into main Oct 11, 2023
1 of 3 checks passed
@30stomercury 30stomercury deleted the asr_whisper_temp0 branch October 11, 2023 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants