forked from kserve/kserve
-
Notifications
You must be signed in to change notification settings - Fork 21
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #39 from israel-hdez/os-ci-onboarding
Openshift-ci onboarding These are the needed changes to have openshift-ci running the E2E tests successfully. There are several groups of E2E tests that can be deduced from the .github/workflows/e2e-test.yaml file: fast, slow, explainer, transformer-mms, qpext, grpc, helm, raw and kourier. For ODH, the `fast`, `slow` and `grpc` groups are the ones that cover the features that are going to be supported in the initial adoption of ODH. This commit contains the needed adaptations to the E2E tests of the `fast` and `slow` groups to successfully run them in an openshift cluster. It also adds a few scripts on test/scripts/openshift-ci to run these E2Es in the openshift-ci operator. Some of these changes should be seen as provisional and should be rolled back: * test/e2e/common/utils.py: because of the networking/DNS expectations, that are currently not covered in ODH's installation. These changes should be rolled back once the following ticked is fixed: opendatahub-io/odh-model-controller#59 * test/e2e/predictor/*: * In general all changes under this path should be seen as provisional. However, since ODH won't support all ServingRuntimes, it is possible that some of the tests will stay out. * There are some GRPC-related tests marked as skipped. Since this work is not enabling the `grpc` group, a subsequent commit/PR for enabling GRPC E2Es should remove/revert those skip marks. * Also, there are some tests skipped with the `Not testable in ODH at the moment` reason. The root cause of the failure should be investigated to re-enable these tests. * python/kserve/kserve/models/v1beta1_inference_service.py: This is injecting an annotation that is required given the specifics of OSSM/Maistra and OpenShift-Serverless that are used in ODH. This annotation is, currently, user responsibility and this was the cleanest way to add it in the E2Es. Being platform-specific, it's been discussed that this (and some other) annotation should be injected by some controller to relief the user from this responsibility. If this happens, this change should be reverted. Also, ideally, changes to the following files should be contributed back to upstream. Those changes are not required in upstream and should have no effect, but in openshift-ci become required because a different builder image is being used: * Dockerfile * agent.Dockerfile
- Loading branch information
Showing
12 changed files
with
473 additions
and
11 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,160 @@ | ||
#!/usr/bin/env bash | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
set -eu | ||
|
||
waitforpodlabeled() { | ||
local ns=${1?namespace is required}; shift | ||
local podlabel=${1?pod label is required}; shift | ||
|
||
echo "Waiting for pod -l $podlabel to be created" | ||
until oc get pod -n "$ns" -l $podlabel -o=jsonpath='{.items[0].metadata.name}' >/dev/null 2>&1; do | ||
sleep 1 | ||
done | ||
} | ||
|
||
waitpodready() { | ||
local ns=${1?namespace is required}; shift | ||
local podlabel=${1?pod label is required}; shift | ||
|
||
waitforpodlabeled "$ns" "$podlabel" | ||
echo "Waiting for pod -l $podlabel to become ready" | ||
oc wait --for=condition=ready --timeout=180s pod -n $ns -l $podlabel | ||
} | ||
|
||
|
||
# Deploy Distributed tracing operator (Jaeger) | ||
cat <<EOF | oc apply -f - | ||
apiVersion: v1 | ||
kind: Namespace | ||
metadata: | ||
name: openshift-distributed-tracing | ||
--- | ||
apiVersion: operators.coreos.com/v1 | ||
kind: OperatorGroup | ||
metadata: | ||
name: openshift-distributed-tracing | ||
namespace: openshift-distributed-tracing | ||
spec: | ||
upgradeStrategy: Default | ||
--- | ||
apiVersion: operators.coreos.com/v1alpha1 | ||
kind: Subscription | ||
metadata: | ||
name: jaeger-product | ||
namespace: openshift-distributed-tracing | ||
spec: | ||
channel: stable | ||
installPlanApproval: Automatic | ||
name: jaeger-product | ||
source: redhat-operators | ||
sourceNamespace: openshift-marketplace | ||
EOF | ||
|
||
waitpodready "openshift-distributed-tracing" "name=jaeger-operator" | ||
|
||
# Deploy Kiali operator | ||
cat <<EOF | oc apply -f - | ||
apiVersion: operators.coreos.com/v1alpha1 | ||
kind: Subscription | ||
metadata: | ||
name: kiali-ossm | ||
namespace: openshift-operators | ||
spec: | ||
channel: stable | ||
installPlanApproval: Automatic | ||
name: kiali-ossm | ||
source: redhat-operators | ||
sourceNamespace: openshift-marketplace | ||
EOF | ||
|
||
waitpodready "openshift-operators" "app=kiali-operator" | ||
|
||
# Deploy OSSM operator | ||
cat <<EOF | oc apply -f - | ||
apiVersion: operators.coreos.com/v1alpha1 | ||
kind: Subscription | ||
metadata: | ||
name: servicemeshoperator | ||
namespace: openshift-operators | ||
spec: | ||
channel: stable | ||
installPlanApproval: Automatic | ||
name: servicemeshoperator | ||
source: redhat-operators | ||
sourceNamespace: openshift-marketplace | ||
EOF | ||
|
||
waitpodready "openshift-operators" "name=istio-operator" | ||
|
||
# Install OSSM | ||
cat <<EOF | oc apply -f - | ||
apiVersion: v1 | ||
kind: Namespace | ||
metadata: | ||
name: istio-system | ||
--- | ||
apiVersion: maistra.io/v2 | ||
kind: ServiceMeshControlPlane | ||
metadata: | ||
name: basic | ||
namespace: istio-system | ||
spec: | ||
version: v2.3 | ||
tracing: | ||
type: Jaeger | ||
sampling: 10000 | ||
# Uncomment if on ROSA | ||
# security: | ||
# identity: | ||
# type: ThirdParty | ||
addons: | ||
jaeger: | ||
name: jaeger | ||
install: | ||
storage: | ||
type: Memory | ||
kiali: | ||
enabled: true | ||
name: kiali | ||
grafana: | ||
enabled: true | ||
EOF | ||
|
||
# Waiting for OSSM minimum start | ||
waitpodready "istio-system" "app=istiod" | ||
|
||
# Create SMMR to enroll namespaces via a label. Also, set mTLS policy to strict by default. | ||
cat <<EOF | oc apply -f - | ||
apiVersion: maistra.io/v1 | ||
kind: ServiceMeshMemberRoll | ||
metadata: | ||
name: default | ||
namespace: istio-system | ||
spec: | ||
memberSelectors: | ||
- matchLabels: | ||
testing.kserve.io/add-to-mesh: "true" | ||
--- | ||
apiVersion: security.istio.io/v1beta1 | ||
kind: PeerAuthentication | ||
metadata: | ||
name: default | ||
namespace: istio-system | ||
spec: | ||
mtls: | ||
mode: STRICT | ||
EOF | ||
|
||
echo -e "\n OSSM has partially started and should be fully ready soon." |
Oops, something went wrong.