how to reproduces openai v3 embedding mteb benchmark ?

[akashAD98]

can you please give us some guide for this?

[Muennighoff]

Maybe the OAI team is interested in providing a guide for reproducing / their script; I didn't run the OAI v3 benchmarking unfortunately cc @Rabrg

[akashAD98]

@Muennighoff bcz i wanted to test the v3-large model with reducing the dimensinality & its impact on mteb

[Muennighoff]

Note that the v3 model is on the leaderboard both with 3072 & with 256 dimensions https://huggingface.co/spaces/mteb/leaderboard

We will also add its other dimensionalities if the OAI team wants to

[akashAD98]

@Muennighoff thanks .looking for OpenAI v3 embedding notebook reproduce script

[akashAD98]

Note that the v3 model is on the leaderboard both with 3072 & with 256 dimensions https://huggingface.co/spaces/mteb/leaderboard

We will also add its other dimensionalities if the OAI team wants to

I'm currently experiencing difficulties in reproducing results with an older version of the OpenAI model. I was wondering if switching to a newer model version could potentially solve these issues. Unfortunately, I haven't had the chance to run the OAI v3 benchmarking script myself. Given this, I am considering trying out the latest model version, but I am uncertain if this would yield successful results