Recommendations for hosting multiple embedding models?

[jordanparker6]

Can TEI handle having multiple models pre-loaded into the container in an environment like cloud run?

Or would it be better to have different TEI services for different models and then handle routing to each.

[OlivierDehaene]

One process of TEI is always responsible for one and only one model. You will need to have a different process to handle routing to the different models.

Now, that doesn't necessarily mean multiple containers.
You could modify the official container to launch multiple processes of TEI inside the container and have nginx or something similar running in the same container to handle the routing and then manage all these processes with supervisord.