Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于DeepSeek-Coder-V2-Lite-Base的128k捞针测试结果 #59

Open
chaochen99 opened this issue Jun 21, 2024 · 1 comment
Open

关于DeepSeek-Coder-V2-Lite-Base的128k捞针测试结果 #59

chaochen99 opened this issue Jun 21, 2024 · 1 comment

Comments

@chaochen99
Copy link

chaochen99 commented Jun 21, 2024

您好,

我测试了deepseek-ai/DeepSeek-Coder-V2-Lite-Base 在128k捞针任务上的表现,结果的正确率不足50%。并且受限于硬件,我无法在deepseek-ai/DeepSeek-Coder-V2-Base 上进行1k到128k的捞针测试。

不知是DeepSeek-Coder-V2-Lite-Base的捞针任务表现一般还是我的测试代码有问题,您可否提供捞针测试代码以便于我重新测试?

感谢!

@hensiesp32
Copy link

hensiesp32 commented Jun 24, 2024

您好,请问您在测试海底捞针的过程中,是否遇到过模型输出不达到max-new-token就不停止的情况呢?
类似于#60 这个问题。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants