Signed-off-by: jooho <jlee@redhat.com>

* generate docs

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* fix

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* fix

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

---------

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

1. graph poetry project was not included in previous checks.
    2. As poetry lock --check is deprecated, used `poetry check --lock`.

Signed-off-by: Andrews Arokiam <andrews.arokiam@ideas2it.com>

* initial commit for graph raw deployment

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* addd hpa support as well for inference graph raw deployment

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Just for local

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Just local

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Fix local setup

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Sleep change

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Fix self-signed-ca installation (#3165)

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* refactored

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Fix logging message

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* adding unit tests

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* adding unit tests

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* removed temporary dev env changes

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Rawdeployment mode type

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* restoring from master branch

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Remove affinity test

Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* add with resource test

Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* added controller test for graph

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* addressed code review comments

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Added copy right statement for new files

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Removed dependency for InferenceGraph kind on componentExtensionSpec which is specific to inference service spec

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* restored Makefile from master branch

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* fixed codescan issue for AtoI function

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* fixed test failures

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Added unit test for hpa reconciler

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Added Ready status when Deployment is available in raw mode

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Removed unused code

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* e2e test for ISVC in raw mode

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* improved e2e tests for inference graph raw deployment

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* fixed python lint errors

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* added annotations for ig spec that triggers raw deployment

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* made names unique b/w runs

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* Add test

Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* Adding unit test

Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>

* fixed e2e test failure for ig.  IG is not moving to ready state because of not handling raw deployment mode condition correctly

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* changed marker for graph test

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* corrected test validation for raw deployment mode knative resources

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* correct rebase errors

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* hpa field moved from annotations to inferencegraphspec as fields

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

* commiting make generate output due to InferenceGraphSpec changes

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>

---------

Signed-off-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Signed-off-by: Tanvi Thakur <tanvi_thakur@intuit.com>
Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>
Co-authored-by: Mopuri, Bharath <bharath_mopuri@intuit.com>
Co-authored-by: Tanvi Thakur <tanvi_thakur@intuit.com>
Co-authored-by: Sivanantham <90966311+sivanantha321@users.noreply.github.com>

When Istio is installed with its CNI plugin, KServe inference services are not capable to start. This is because the storage initializer is an init-container and the network is not available when the CNI plugin is enabled.

The typical recommendation to fix the issue is to remove init-containers and move any initialization code to a regular container. This approach would not work well with KServe, because the serving runtimes assume the model is already present on the filesystem and moving the storage initializer as a regular container will cause race conditions (the runtime will succeed loading only if the storage initializer manages to pull the model before the runtime starts).

There are alternative approaches documented in https://istio.io/latest/docs/setup/additional-setup/cni/#compatibility-with-application-init-containers. All alternatives have the downside that the traffic won't be captured by Istio and won't benefit from Istio features, which should be OK for KServe storage-initializer case.

These changes use the approach for running the storage initializer using the same UserID as the Istio sidecar. The UID is copied from the sidecar container to cover Istio derivatives, like Maistra.

Signed-off-by: Edgar Hernández <23639005+israel-hdez@users.noreply.github.com>

* Pass missing infer parameters

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Add tests

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Fix warnings

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Add equal magic method for infer types

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* Avoid inplace modification of parameters in conversion

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

---------

Signed-off-by: Sivanantham Chinnaiyan <sivanantham.chinnaiyan@ideas2it.com>

* feat: Support S3 transfer acceleration

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* Add missing field in ISVC yaml and storage.py

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* fix test

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* fix default

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

---------

Signed-off-by: Yuan Tang <terrytangyuan@gmail.com>

* Initial huggingface server implementation

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Implement preprocess and predict for huggingface server

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add postprocess

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Update lock file

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Update func signature

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Update encoding

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add token classification test

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add comments

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix comments

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add generate endpoint

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Remove vllm async engine file

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Handle optional vllm

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add async generator

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix lint

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Generate response streaming support

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* rename async generate stream file

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Call generate endpoint directly

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Remove the tritonclient dependency

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add huggingface dockerfile

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Test for triton bert case

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add huggingface test

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add Makefile

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Support gpu device

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Add gnereate parameters

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Fix generate tests

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

* Run huggingface test

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

---------

Signed-off-by: Dan Sun <dsun20@bloomberg.net>

…elcars (#3384)

Modelcar: Add mount of shared volume to transformer container

Adding the missing mount of the emptyDir volume that holds the symbolic link to the model data in the model sidecar to a transformer container, if it is configured.

Signed-off-by: Roland Huß <rhuss@redhat.com>

chore:  The cloudevents/sdk-go was migrated to v2, however, the v1 is still present in the
        go.mod. It was being used bu the pkg/logger/worker.go file to set a field that was
        not being in use anymore, thus it was removed

Signed-off-by: Spolti <fspolti@redhat.com>