[NOTE] You can also customize the "LVM_MODEL_ID" if needed.
You need to make sure you have created the directory
/mnt/opea-models
to save the cached model on the node where the visualqna workload is running. Otherwise, you need to modify thevisualqna.yaml
file to change themodel-volume
to a directory that exists on the node.
cd GenAIExamples/visualqna/kubernetes/intel/cpu/xeon/manifests
kubectl apply -f visualqna.yaml
cd GenAIExamples/visualqna/kubernetes/intel/hpu/gaudi/manifests
kubectl apply -f visualqna.yaml
To verify the installation, run the command kubectl get pod
to make sure all pods are running.
Then run the command kubectl port-forward svc/visualqna 8888:8888
to expose the visualqna service for access.
Open another terminal and run the following command to verify the service if working:
curl http://localhost:8888/v1/visualqna \
-H 'Content-Type: application/json' \
-d '{"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "What'\''s in this image?"
},
{
"type": "image_url",
"image_url": {
"url": "https://www.ilankelman.org/stopsigns/australia.jpg"
}
}
]
}
],
"max_tokens": 128}'