add k8s docs for getting started and helm

[devpramod]

This PR contains the following docs:

• Getting Started for k8s - Installation, basic introduction to k8s and has a section for helm. As more k8s deployment modes are added, corresponding sections will be created in this doc

• Deploy using helm charts, a doc that follows the xeon.md template as much as possible to deploy ChatQnA on k8s

[dbkinder]

some suggested edits

Also, when you add new documents, they need to be linked into the table of contents structure. There's an index.rst file in this folder you can edit to add these two documents.

I'd suggest you add an edit to the index.rst doc in this deploy folder, and replace the existing Kubernetes section with this:

Kubernetes
**********

.. toctree::
   :maxdepth: 1

   k8s_getting_started
   TGI on Xeon with Helm Charts <k8s_helm>

* Xeon & Gaudi with GMC
* Xeon & Gaudi without GMC

[dbkinder]

LGTM, thanks!

[tylertitsworth]

Some of the stuff I see in the docs, is just a tutorial on things that already have docs. Like TGI/TEI, Helm, and Kubernetes. It feels a lot like we're overexplaining a concept that can be answered by a link to the source docs of another tool and a command for how it's relevant to use with ChatQnA.

For reference, this is the most handholding I would do in the case of deploying TGI:

---

Configure Model Server

Before we deploy a model, we need to configure the model server with information like, what model to use and how many max tokens to use. We will be using the tgi-on-intel helm chart. This chart uses XPU to the serve model normally, but we are going to configure it to use gaudi2 instead.

First, look at the configuration files in the tgi directory and add/remove any configuration options relevant to your workflow:

cd tgi
# Create a new configmap for your model server to use
kubectl apply -f cm.yaml

Tip

Here is the reference to the Huggingface Launcher Environment Variables and the TGI-Gaudi Environment Variables.

Deploy Model Server

Now that we have configured the model server, we can deploy it to Kubernetes. Using the provided config.yaml file in the tgi directory, we can deploy the model server.

Modify any values like resources or replicas in the config.yaml file to suit your needs. Then, deploy the model server:

# Encode HF Token for secret.encodedToken
echo -n '<token>' | base64
# Install Chart
git clone https://github.com/intel/ai-containers
helm install model-server -f config.yaml ai-containers/workflows/charts/tgi
# Check the pod status
kubectl get pod
kubectl logs -f <pod-name>

---

Please use a tool like markdownlint to ensure consistent styling.

[tylertitsworth]

There is a very subtle difference between bolded and non-bolded text that is confusing to read. I would rather see an h4 here (####) than bolded text + a colon.

[devpramod]

@tylertitsworth
Do you suggest switching to an h4 only for the section "Kubernetes Cluster and Development Environment"?
reason I ask is, the document has many items that are bolded and may not be suitable for h4
like kubectl, pods, Using Helm Charts, Using Manifests etc

[tylertitsworth]

No installation notes is a bit strange, this could do a LOT of heavy lifting for the docs here.

[devpramod]

@tylertitsworth The installation notes are under Validate microservice --> Check the pod status which are the following sections
Do you have any suggestions to make it better? or add something?

[dbkinder]

I've got a script in docs/scripts/checkmd.sh that uses pymarkdown (lint) to scan markdown files, with a bunch of checks disabled. Alas, if I wasn't retiring today, including a markdown linter was on my list to add to the CI checks. :)