Change in domain templates

[bartoszmajsak]

/kind feature

Describe the solution you'd like
We would like to have an enhancement to the domain template used in KServe. Currently, it's defined as {{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }}.

Changing it to {{ .Name }}.{{ .Namespace }}.{{ .IngressDomain }} will make the AuthConfigs more specific, allowing for the creation of a configuration for *.{{.Namespace}} directly instead of monitoring all models and creating new "hosts".

[bartoszmajsak]

/cc @aslakknutsen

[Jooho]

@bartoszmajsak To be clear what we should do with this ticket, I have some questions.
is this request to change this configuraton:https://github.com/opendatahub-io/kserve/blob/master/config/configmap/inferenceservice.yaml#L181 to {{ .Name }}.{{ .Namespace }}.{{ .IngressDomain }} for AuthConfig?

[bartoszmajsak]

@bartoszmajsak To be clear what we should do with this ticket, I have some questions. is this request to change this configuraton:master/config/configmap/inferenceservice.yaml#L181 to {{ .Name }}.{{ .Namespace }}.{{ .IngressDomain }} for AuthConfig?

Technically speaking more about this

kserve/config/configmap/inferenceservice.yaml

Line 451 in b6eac44

"domainTemplate": "{{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }}",

@aslakknutsen can you confirm?

[aslakknutsen]

domainTemplateis part of the KServe configuration. On the Authorino side we canonly bind AuthConfigs to "hosts" with wildcard levels(*.x.y), not partly wildcards(*-x.y).

This change would allow us to handle 1 domain pattern between all models in a odh-project(namespace) without the need for additional listeners to update the host lists or filters to manipulate the incoming host name.

[aslakknutsen]

@aslakknutsen can you confirm?

Not 100% sure of the source of this configuration in the kserve world and how that is moved to osd-manifests etc, but that certainly would look like it.

[Jooho]

@vaibhavjainwiz @israel-hdez
If we change this, will there be any problems? I think this PR might need changes. What do you think?

[israel-hdez]

@aslakknutsen @bartoszmajsak Do you need to change only the InferenceService public domain? Or would you also need a similar change to the domain of the KNative services?

[aslakknutsen]

@israel-hdez Not sure what the KNative services are in this context? Would that be like the "internal" names?

[israel-hdez]

@aslakknutsen In KServe, when you create an InferenceService, it creates KNative services for you to deploy the models.

The ksvcs can also be publicly exposed with their own URL with a similar pattern that also seems to be {{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }} by default.

So, several routes can be involved:

1. The route for the ISVC itself.
2. But we can also have several KNative routes.

My understanding is that you would want (1) changed, but not sure if routes in (2) would be relevant.

In any case, I was looking at doc comments in the inferenceservice-config ConfigMap, and it looks like the template is only applicable to RawDeployment which, at the moment, we are not using. See:

kserve/config/configmap/inferenceservice.yaml

Lines 215 to 223 in 799e121

	# domainTemplate specifies the template for generating domain/url for each inference service by combining variable from:
	# Name of the inference service ( {{ .Name}} )
	# Namespace of the inference service ( {{ .Namespace }} )
	# Annotation of the inference service ( {{ .Annotations.key }} )
	# Label of the inference service ( {{ .Labels.key }} )
	# IngressDomain ( {{ .IngressDomain }} )
	# If domain template is empty the default template {{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }} is used.
	# NOTE: This configuration only applicable for raw deployment.
	"domainTemplate": "{{ .Name }}-{{ .Namespace }}.{{ .IngressDomain }}",

.

I haven't tested, but from those doc comments, my understanding is that changing that template won't have any impact on the generated URLs for KServe Serverless installation. It seems that KNative is the component that is controlling the URL template. However, I couldn't find any docs for configuring KNative's URL template.

So, my perception is that is not as easy as changing the template. It may be an enhancement that needs to be done to KServe.

[israel-hdez]

Closing as no longer needed.