Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add k8s docs for getting started and helm #179

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

devpramod
Copy link
Contributor

This PR contains the following docs:

  • Getting Started for k8s - Installation, basic introduction to k8s and has a section for helm. As more k8s deployment modes are added, corresponding sections will be created in this doc

  • Deploy using helm charts, a doc that follows the xeon.md template as much as possible to deploy ChatQnA on k8s

Signed-off-by: devpramod <pramod.pai@intel.com>
Copy link
Contributor

@dbkinder dbkinder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some suggested edits

Also, when you add new documents, they need to be linked into the table of contents structure. There's an index.rst file in this folder you can edit to add these two documents.

I'd suggest you add an edit to the index.rst doc in this deploy folder, and replace the existing Kubernetes section with this:

Kubernetes
**********

.. toctree::
   :maxdepth: 1

   k8s_getting_started
   TGI on Xeon with Helm Charts <k8s_helm>

* Xeon & Gaudi with GMC
* Xeon & Gaudi without GMC

examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
Signed-off-by: devpramod <pramod.pai@intel.com>

Signed-off-by: devpramod <pramod.pai@intel.com>
examples/ChatQnA/deploy/index.rst Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Show resolved Hide resolved
Signed-off-by: devpramod <pramod.pai@intel.com>
Copy link
Contributor

@dbkinder dbkinder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link

@tylertitsworth tylertitsworth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the stuff I see in the docs, is just a tutorial on things that already have docs. Like TGI/TEI, Helm, and Kubernetes. It feels a lot like we're overexplaining a concept that can be answered by a link to the source docs of another tool and a command for how it's relevant to use with ChatQnA.

For reference, this is the most handholding I would do in the case of deploying TGI:


Configure Model Server

Before we deploy a model, we need to configure the model server with information like, what model to use and how many max tokens to use. We will be using the tgi-on-intel helm chart. This chart uses XPU to the serve model normally, but we are going to configure it to use gaudi2 instead.

First, look at the configuration files in the tgi directory and add/remove any configuration options relevant to your workflow:

cd tgi
# Create a new configmap for your model server to use
kubectl apply -f cm.yaml

Tip

Here is the reference to the Huggingface Launcher Environment Variables and the TGI-Gaudi Environment Variables.

Deploy Model Server

Now that we have configured the model server, we can deploy it to Kubernetes. Using the provided config.yaml file in the tgi directory, we can deploy the model server.

Modify any values like resources or replicas in the config.yaml file to suit your needs. Then, deploy the model server:

# Encode HF Token for secret.encodedToken
echo -n '<token>' | base64
# Install Chart
git clone https://github.com/intel/ai-containers
helm install model-server -f config.yaml ai-containers/workflows/charts/tgi
# Check the pod status
kubectl get pod
kubectl logs -f <pod-name>

Please use a tool like markdownlint to ensure consistent styling.

examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved

### Kubernetes Cluster and Development Environment

**Setting Up the Kubernetes Cluster:** Before beginning deployment for the ChatQnA application, ensure that a Kubernetes cluster is ready. For guidance on setting up your Kubernetes cluster, please refer to the comprehensive setup instructions available on the [Opea Project deployment guide](https://opea-project.github.io/latest/deploy/index.html).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a very subtle difference between bolded and non-bolded text that is confusing to read. I would rather see an h4 here (####) than bolded text + a colon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tylertitsworth
Do you suggest switching to an h4 only for the section "Kubernetes Cluster and Development Environment"?
reason I ask is, the document has many items that are bolded and may not be suitable for h4
like kubectl, pods, Using Helm Charts, Using Manifests etc

examples/ChatQnA/deploy/k8s_getting_started.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
NAMESPACE: chatqa
STATUS: deployed
REVISION: 1

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No installation notes is a bit strange, this could do a LOT of heavy lifting for the docs here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tylertitsworth The installation notes are under Validate microservice --> Check the pod status which are the following sections
Do you have any suggestions to make it better? or add something?

examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Outdated Show resolved Hide resolved
examples/ChatQnA/deploy/k8s_helm.md Show resolved Hide resolved
@dbkinder
Copy link
Contributor

I've got a script in docs/scripts/checkmd.sh that uses pymarkdown (lint) to scan markdown files, with a bunch of checks disabled. Alas, if I wasn't retiring today, including a markdown linter was on my list to add to the CI checks. :)

Signed-off-by: devpramod <pramod.pai@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants