-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add script to wait for the model to appear when using KServe Modelcar #356
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this additional script really necessary? Shouldn't the storage initializer handle this? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The problem is that the storage initializer runs as an init-container which runs before the modelcar sidecar starts in parallel with the runtime container. The proper solution is to add the modelcar as a proper K8s sidecar as described in kserve/kserve#3646, but this requires K8s >= 1.29 + this feature needs to be enabled. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
#!/bin/bash | ||
|
||
if [ "${MODEL_INIT_MODE}" = "async" ] ; then | ||
echo "Waiting for model files (modelcar) to be present..." | ||
until test -e /mnt/models; do | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It might be worth to also add a timeout, as there can be also error case when the modelcar can't be started. Of course, the overall Pod will be in an error state if the modelcar can't be fired up (e.g. when specifying a non-existing image as |
||
sleep 1 | ||
done | ||
|
||
echo "Model files are now available." | ||
fi | ||
|
||
echo "Starting model server..." | ||
eval $@ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are the semantics here of changing the entrypoint? Does CMD run afterwards? I was under the impression that the entrypoint is what's run when the container starts up
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When an ENTRYPOINT is specified, then the CMD becomes its arguments (the separation comes from the early Docker days, so that the image author could specify a command with ENTRYPOINT that could not be overwritten with
docker run
). In the Kubernetes world, ENTRYPOINT correspond to thecommand:
field and CMD correspond to theargs:
field in a Pod container's spec.