Skip to content

Commit

Permalink
deploy: f01819d
Browse files Browse the repository at this point in the history
  • Loading branch information
NeuralChatBot committed Oct 28, 2024
1 parent f01819d commit b381b2f
Show file tree
Hide file tree
Showing 178 changed files with 24,547 additions and 0 deletions.
Binary file added latest/_images/BrokenBlocks.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/Edge_Craft_RAG.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/Edge_Craft_RAG_screenshot_1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/Edge_Craft_RAG_screenshot_2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/Login_page.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/MultimodalQnA.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/add_group_scope.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/add_mapper.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/agent_arch.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/agent_qna_arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/attach_group_scope.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/audio_ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/audio_ui_record.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/avatars-chatbot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/avatars-ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chat_qna_init.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chat_ui_init.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chat_ui_response.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chat_ui_response1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chat_ui_upload.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chathistory_flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chatqna_architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added latest/_images/chatqna_dashboards.png
Binary file added latest/_images/chatqna_flow_chart.png
Binary file added latest/_images/chatqna_with_conversation.png
Binary file added latest/_images/codeGen_ui_init.jpg
Binary file added latest/_images/codeGen_ui_response.png
Binary file added latest/_images/codeTrans_ui_init.png
Binary file added latest/_images/codeTrans_ui_response.png
Binary file added latest/_images/codeTrans_ui_select.png
Binary file added latest/_images/code_trans_architecture.png
Binary file added latest/_images/codegen.png
Binary file added latest/_images/codegen_architecture.png
Binary file added latest/_images/codegen_auto_complete.jpg
Binary file added latest/_images/codegen_copilot.png
Binary file added latest/_images/codegen_customize.png
Binary file added latest/_images/codegen_dialog.png
Binary file added latest/_images/codegen_endpoint.png
Binary file added latest/_images/codegen_icon.png
Binary file added latest/_images/codegen_multi_line.png
Binary file added latest/_images/codegen_qna.png
Binary file added latest/_images/codegen_react.png
Binary file added latest/_images/codegen_select_code.png
Binary file added latest/_images/codegen_settings.png
Binary file added latest/_images/codegen_single_line.png
Binary file added latest/_images/codegen_suggestion.png
Binary file added latest/_images/conversation_ui_init.png
Binary file added latest/_images/conversation_ui_response.png
Binary file added latest/_images/conversation_ui_upload.png
Binary file added latest/_images/create_client.png
Binary file added latest/_images/create_client_1.png
Binary file added latest/_images/create_client_2.png
Binary file added latest/_images/create_client_3.png
Binary file added latest/_images/create_productivitysuite_realm.png
Binary file added latest/_images/create_realm.png
Binary file added latest/_images/create_realm1.png
Binary file added latest/_images/create_roles.png
Binary file added latest/_images/create_user.png
Binary file added latest/_images/create_users.png
Binary file added latest/_images/data_source.png
Binary file added latest/_images/dbQnA_ui_db_credentials.png
Binary file added latest/_images/dbQnA_ui_enter_question.png
Binary file added latest/_images/dbQnA_ui_failed_db_connection.png
Binary file added latest/_images/dbQnA_ui_init.png
Binary file added latest/_images/design.png
Binary file added latest/_images/docSum_ui_exchange.png
Binary file added latest/_images/docSum_ui_response.png
Binary file added latest/_images/docSum_ui_text.png
Binary file added latest/_images/docSum_ui_upload.png
Binary file added latest/_images/doc_summary_file.png
Binary file added latest/_images/doc_summary_paste.png
Binary file added latest/_images/docsum-ui-react-error.png
Binary file added latest/_images/docsum-ui-react-file.png
Binary file added latest/_images/docsum-ui-react.png
Binary file added latest/_images/docsum_architecture.png
Binary file added latest/_images/docsum_workflow.png
Binary file added latest/_images/example_query.png
Binary file added latest/_images/faq_generator.png
Binary file added latest/_images/faqgen_react_ui_text.png
Binary file added latest/_images/faqgen_react_ui_text_file.png
Binary file added latest/_images/faqgen_ui_text.png
Binary file added latest/_images/framework-image1.png
Binary file added latest/_images/framework-image10.png
Binary file added latest/_images/framework-image11.png
Binary file added latest/_images/framework-image12.png
Binary file added latest/_images/framework-image13.png
Binary file added latest/_images/framework-image14.png
Binary file added latest/_images/framework-image15.png
Binary file added latest/_images/framework-image16.png
Binary file added latest/_images/framework-image17.png
Binary file added latest/_images/framework-image2.png
Binary file added latest/_images/framework-image3.png
Binary file added latest/_images/framework-image4.png
Binary file added latest/_images/framework-image5.png
Binary file added latest/_images/framework-image6.png
Binary file added latest/_images/framework-image7.png
Binary file added latest/_images/framework-image9.png
Binary file added latest/_images/grafana_dashboard.png
Binary file added latest/_images/image-1.png
Binary file added latest/_images/image-2.png
Binary file added latest/_images/image.png
Binary file added latest/_images/image_wav2lipgfpgan_cut.gif
Binary file added latest/_images/keycloak_login.png
Binary file added latest/_images/llava_screenshot1.png
Binary file added latest/_images/llava_screenshot2.png
Binary file added latest/_images/multimodal_enhanced_audio_ui.png
Binary file added latest/_images/multimodal_enhanced_chat_ui.png
Binary file added latest/_images/multimodal_enhanced_diagram.png
Binary file added latest/_images/multimodal_enhanced_image_ui.png
Binary file added latest/_images/multimodal_enhanced_pdf_ui.png
Binary file added latest/_images/multimodal_enhanced_video_ui.png
Binary file added latest/_images/opea-docs-fork.png
Binary file added latest/_images/opea_architecture.png
Binary file added latest/_images/opea_workflow.png
Binary file added latest/_images/personas.png
Binary file added latest/_images/reranking_flow.png
Binary file added latest/_images/search_ui_init.png
Binary file added latest/_images/searchqna.png
Binary file added latest/_images/set_user_password.png
Binary file added latest/_images/text2img_init.png
Binary file added latest/_images/text2img_response.png
Binary file added latest/_images/tgi_dashboard.png
Binary file added latest/_images/trans_ui_init.png
Binary file added latest/_images/trans_ui_select.png
Binary file added latest/_images/translation_architecture.png
Binary file added latest/_images/upload-gen-captions.png
Binary file added latest/_images/upload-gen-trans.png
Binary file added latest/_images/user_role_mapping.png
Binary file added latest/_images/video_wav2lipgfpgan_cut.gif
Binary file added latest/_images/videoqna.gif
Binary file added latest/_images/vqa.png
153 changes: 153 additions & 0 deletions latest/_sources/getting-started/README.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# Getting Started with OPEA
This document specifically details the steps for deploying services on IBM Cloud, providing a tailored guide to help you leverage IBM's cloud infrastructure to deploy the ChatQnA application from OPEA GenAI Examples. For additional deployment targets, see the [ChatQnA Sample Guide](https://opea-project.github.io/latest/examples/ChatQnA/ChatQnA_Guide.html)

## Understanding OPEA's Core Components

Before moving forward, it's important to familiarize yourself with two key elements of OPEA: GenAIComps and GenAIExamples.

- GenAIComps is a collection of microservice components that form a service-based toolkit. This includes a variety of services such as llm (large language models), embedding, and reranking, among others.
- While GenAIComps offers a range of microservices, GenAIExamples provides practical, deployable solutions to help users implement these services effectively. Examples include ChatQnA and DocSum, which leverage the microservices for specific applications.


## Prerequisites


## Create and Configure a Virtual Server
1. Navigate to [IBM Cloud](https://cloud.ibm.com). - Click the **Create resource** button at the top right of the screen. Select **Compute** from the options available and select `Virtual Server for VPC`

2. Select a location for the instance. Assign a name to it.

3. Under Stock Images, select Ubuntu 24.04 (`ibm-ubuntu-24-04-6-minimal-amd64-1`)

4. Select a virtual server.
> **Note:** We recommend selecting a 3-series instance with an Intel(R) 4th Gen Xeon(C) Scalable Processor, such as `bx3d-16x80` or above. For more information on virtual servers on IBM cloud visit [Intel® solutions on IBM Cloud®](https://www.ibm.com/cloud/intel).

5. Add an SSH key to the instance, if necessary, create one first.

6. Click on `Create virtual server`.

7. Once the instance is running, create and attach a `Floating IP` to the instance. For more information visit [this](https://cloud.ibm.com/docs/vpc?topic=vpc-fip-working&interface=ui) site

8. `ssh` into the instance using the floating IP (`ssh -i <key> ubuntu@<floating-ip>`)



## Deploy the ChatQnA Solution
Use the command below to install docker on a clean virtual machine
```bash
wget https://raw.githubusercontent.com/opea-project/GenAIExamples/refs/heads/main/ChatQnA/docker_compose/install_docker.sh
chmod +x install_docker.sh
./install_docker.sh
```
Configure Docker to run as a non-root user by following these [instructions](https://docs.docker.com/engine/install/linux-postinstall/)

Clone the repo and navigate to ChatQnA

```bash
git clone https://github.com/opea-project/GenAIExamples.git
cd GenAIExamples/ChatQnA
```
Set the required environment variables:
```bash
export host_ip="localhost"
export HUGGINGFACEHUB_API_TOKEN="Your_Huggingface_API_Token"
```

Set up other specific use-case environment variables:
```bash
cd docker_compose/intel/cpu/xeon/
source set_env.sh
```
Now we can start the services
```bash
docker compose up -d
```
It takes a few minutes for the services to start. Check the logs for the services to ensure that ChatQnA is running.
For example to check the logs for the `tgi-service`:

```bash
docker compose logs tgi-service | grep Connected
```
The output shows `Connected` as shown:
```
tgi-service | 2024-10-18T22:41:18.973042Z INFO text_generation_router::server: router/src/server.rs:2311: Connected
```

Run `docker ps -a` as an additional check to verify that all the services are running as shown:

```
| CONTAINER ID | IMAGE | COMMAND | CREATED | STATUS | PORTS | NAMES |
|--------------|------------------------------------------------------------------------|------------------------|--------------|-------------|------------------------------------------------------------------------------------------|------------------------------|
| 3a65ff9e16bd | opea/nginx:latest | `/docker-entrypoint.\…`| 14 hours ago | Up 14 hours | 0.0.0.0:80->80/tcp, :::80->80/tcp | chatqna-xeon-nginx-server |
| 7563b2ee1cd9 | opea/chatqna-ui:latest | `docker-entrypoint.s\…`| 14 hours ago | Up 14 hours | 0.0.0.0:5173->5173/tcp, :::5173->5173/tcp | chatqna-xeon-ui-server |
| 9ea57a660cd6 | opea/chatqna:latest | `python chatqna.py` | 14 hours ago | Up 14 hours | 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp | chatqna-xeon-backend-server |
| 451bacaac3e6 | opea/retriever-redis:latest | `python retriever_re\…`| 14 hours ago | Up 14 hours | 0.0.0.0:7000->7000/tcp, :::7000->7000/tcp | retriever-redis-server |
| c1f952ef5c08 | opea/dataprep-redis:latest | `python prepare_doc_\…`| 14 hours ago | Up 14 hours | 0.0.0.0:6007->6007/tcp, :::6007->6007/tcp | dataprep-redis-server |
| 2a874ed8ce6f | redis/redis-stack:7.2.0-v9 | `/entrypoint.sh` | 14 hours ago | Up 14 hours | 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp, 0.0.0.0:8001->8001/tcp, :::8001->8001/tcp | redis-vector-db |
| ac7b62306eb8 | ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 | `text-embeddings-rou\…`| 14 hours ago | Up 14 hours | 0.0.0.0:8808->80/tcp, [::]:8808->80/tcp | tei-reranking-server |
| 521cc7faa00e | ghcr.io/huggingface/text-generation-inference:sha-e4201f4-intel-cpu | `text-generation-lau\…`| 14 hours ago | Up 14 hours | 0.0.0.0:9009->80/tcp, [::]:9009->80/tcp | tgi-service |
| 9faf553d3939 | ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 | `text-embeddings-rou\…`| 14 hours ago | Up 14 hours | 0.0.0.0:6006->80/tcp, [::]:6006->80/tcp | tei-embedding-server |

```

### Interact with ChatQnA

You can interact with ChatQnA via a browser interface:
* Under `Infrastructure` in the left pane, go to `Network/Security groups/<Your Security Group>/Rules`
* Select `Create`
* Enable inbound traffic for port 80
* To view the ChatQnA interface, open a browser and navigate to the UI by inserting your externally facing IP address in the following: `http://{external_public_ip}:80'.

For more information on editing inbound/outbound rules, click [here](https://cloud.ibm.com/docs/vpc?topic=vpc-updating-the-default-security-group&interface=ui)

A snapshot of the interface looks as follows:

![Chat Interface](assets/chat_ui_response.png)


> **Note:** this example leverages the Nike 2023 Annual report for its RAG based content. See the [ChatQnA Sample Guide](https://opea-project.github.io/latest/examples/ChatQnA/ChatQnA_Guide.html)
to learn how you can customize the example with your own content.

To interact with the ChatQnA application via a `curl` command:

```bash
curl http://${host_ip}:8888/v1/chatqna \
-H "Content-Type: application/json" \
-d '{
"messages": "What is the revenue of Nike in 2023?"
}'
```
ChatQnA provides the answer to your query as a text stream.

Modify the `message` parameter in the `curl` command to continue interacting with ChatQnA.

## What’s Next

- Try [GenAIExamples](/examples/index.rst) in-detail starting with [ChatQnA](/examples/ChatQnA/ChatQnA_Guide.rst) example; this is a great example to orient yourself to the OPEA ecosystem.
- Try [GenAIComps](/microservices/index.rst) to build microservices.

### Get Involved

Have you ideas and skills to build out genAI components, microservices, and solutions? Would you like to be a part of this evolving technology in its early stages? Welcome!
* Register for our mailing list:
* [Mailing List](https://lists.lfaidata.foundation/g/OPEA-announce)
* [Technical Discussions](https://lists.lfaidata.foundation/g/OPEA-technical-discuss)
* Subscribe to the working group mailing lists that interest you
* [End user](https://lists.lfaidata.foundation/g/OPEA-End-User)
* [Evaluation](https://lists.lfaidata.foundation/g/OPEA-Evaluation)
* [Community](https://lists.lfaidata.foundation/g/OPEA-Community)
* [Research](https://lists.lfaidata.foundation/g/OPEA-Research)
* [Security](https://lists.lfaidata.foundation/g/OPEA-Security)
* Go to the Community Section of the OPEA repo for [Contribution Guidelines](https://opea-project.github.io/latest/community/CONTRIBUTING.html) and step by step instructions.
* Attend any of our community events and hackathons. https://wiki.lfaidata.foundation/display/DL/OPEA+Community+Events

Current GenAI Examples
- Simple chatbot that uses retrieval augmented generation (RAG) architecture. [ChatQnA](/examples/ChatQnA/ChatQnA_Guide.rst)
- Code generation, from enabling non-programmers to generate code to improving productivity with code completion of complex applications. [CodeGen](https://opea-project.github.io/latest/GenAIExamples/CodeGen/README.html)
- Make your applications more flexible by porting to different languages. [CodeTrans](https://opea-project.github.io/latest/GenAIExamples/CodeTrans/README.html)
- Create summaries of news articles, research papers, technical documents, etc. to streamline content systems. [DocSum](https://opea-project.github.io/latest/GenAIExamples/DocSum/README.html)
- Mimic human behavior by iteratively searching, selecting, and synthesizing information across large bodies of content. [SearchQnA](https://opea-project.github.io/latest/GenAIExamples/SearchQnA/README.html)
- Provide critical content to your customers by automatically generating Frequently Asked Questions (FAQ) resources. [FaqGen](https://opea-project.github.io/latest/GenAIExamples/FaqGen/README.html)
- Provide text descriptions from pictures, enabling your users to inquire directly about products, services, sites, etc. [VisualQnA](https://opea-project.github.io/latest/GenAIExamples/VisualQnA/README.html)
- Reduce language barriers through customizable text translation systems. [Translation](https://opea-project.github.io/latest/GenAIExamples/Translation/README.html)

10 changes: 10 additions & 0 deletions latest/_sources/glossary.rst.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
:orphan:

.. _glossary:

Glossary of Terms
#################

.. glossary::
:sorted:

119 changes: 119 additions & 0 deletions latest/_sources/guide/installation/gmc_install/gmc_install.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# GenAI-microservices-connector(GMC) Installation

This document will introduce the GenAI Microservices Connector (GMC) and its installation. It will then use the ChatQnA pipeline as a use case to demonstrate GMC's functionalities.

## GenAI-microservices-connector(GMC)

GMC can be used to compose and adjust GenAI pipelines dynamically on Kubernetes. It can leverage the microservices provided by GenAIComps and external services to compose GenAI pipelines. External services might be running in a public cloud or on-prem. Just provide an URL and access details such as an API key and ensure there is network connectivity. It also allows users to adjust the pipeline on the fly like switching to a different Large language Model(LLM), adding new functions into the chain(like adding guardrails), etc. GMC supports different types of steps in the pipeline, like sequential, parallel and conditional. For more information: https://github.com/opea-project/GenAIInfra/tree/main/microservices-connector

## Install GMC

**Prerequisites**

- For the ChatQnA example ensure your cluster has a running Kubernetes cluster with at least 16 CPUs, 32GB of memory, and 100GB of disk space. To install a Kubernetes cluster refer to:
["Kubernetes installation"](../k8s_install/README.md)

**Download the GMC github repository**

```sh
git clone https://github.com/opea-project/GenAIInfra.git && cd GenAIInfra/microservices-connector
```

**Build and push your image to the location specified by `CTR_IMG`:**

```sh
make docker.build docker.push CTR_IMG=<some-registry>/gmcmanager:<tag>
```

**NOTE:** This image will be published in the personal registry you specified.
And it is required to have access to pull the image from the working environment.
Make sure you have the proper permissions to the registry if the above commands don’t work.

**Install GMC CRD**

```sh
kubectl apply -f config/crd/bases/gmc.opea.io_gmconnectors.yaml
```

**Get related manifests for GenAI Components**

```sh
mkdir -p $(pwd)/config/manifests
cp $(dirname $(pwd))/manifests/ChatQnA/*.yaml -p $(pwd)/config/manifests/
```

**Copy GMC router manifest**

```sh
cp $(pwd)/config/gmcrouter/gmc-router.yaml -p $(pwd)/config/manifests/
```

**Create Namespace for gmcmanager deployment**

```sh
export SYSTEM_NAMESPACE=system
kubectl create namespace $SYSTEM_NAMESPACE
```

**NOTE:** Please use the exact same `SYSTEM_NAMESPACE` value setting you used while deploying gmc-manager.yaml and gmc-manager-rbac.yaml.

**Create ConfigMap for GMC to hold GenAI Components and GMC Router manifests**

```sh
kubectl create configmap gmcyaml -n $SYSTEM_NAMESPACE --from-file $(pwd)/config/manifests
```

**NOTE:** The configmap name `gmcyaml' is defined in gmcmanager deployment Spec. Please modify accordingly if you want use a different name for the configmap.

**Install GMC manager**

```sh
kubectl apply -f $(pwd)/config/rbac/gmc-manager-rbac.yaml
kubectl apply -f $(pwd)/config/manager/gmc-manager.yaml
```

**Check the installation result**

```sh
kubectl get pods -n system
NAME READY STATUS RESTARTS AGE
gmc-controller-78f9c748cb-ltcdv 1/1 Running 0 3m
```

## Use GMC to compose a chatQnA Pipeline
A sample for chatQnA can be found at config/samples/chatQnA_xeon.yaml

**Deploy chatQnA GMC custom resource**

```sh
kubectl create ns chatqa
kubectl apply -f $(pwd)/config/samples/chatQnA_xeon.yaml
```

**GMC will reconcile chatQnA custom resource and get all related components/services ready**

```sh
kubectl get service -n chatqa
```

**Check GMC chatQnA custom resource to get access URL for the pipeline**

```bash
$kubectl get gmconnectors.gmc.opea.io -n chatqa
NAME URL READY AGE
chatqa http://router-service.chatqa.svc.cluster.local:8080 8/0/8 3m
```

**Deploy one client pod for testing the chatQnA application**

```bash
kubectl create deployment client-test -n chatqa --image=python:3.8.13 -- sleep infinity
```

**Access the pipeline using the above URL from the client pod**

```bash
export CLIENT_POD=$(kubectl get pod -n chatqa -l app=client-test -o jsonpath={.items..metadata.name})
export accessUrl=$(kubectl get gmc -n chatqa -o jsonpath="{.items[?(@.metadata.name=='chatqa')].status.accessUrl}")
kubectl exec "$CLIENT_POD" -n chatqa -- curl $accessUrl -X POST -d '{"text":"What is the revenue of Nike in 2023?","parameters":{"max_new_tokens":17, "do_sample": true}}' -H 'Content-Type: application/json'
```
8 changes: 8 additions & 0 deletions latest/_sources/guide/installation/k8s_install/README.md.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Kubernetes Installation Options

Here are a variety of ways to install Kubernetes:

* [Using AWS EKS Cluster](k8s_instal_aws_eks.md)
* [Using kubeadm](k8s_install_kubeadm.md)
* [Using Kubespray](k8s_install_kubespray.md)

Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Kubernetes Installation using AWS EKS Cluster

In this document, we'll install Kubernetes v1.30 using [AWS EKS Cluster](https://docs.aws.amazon.com/eks/latest/userguide/clusters.html).


There are two ways to create a new Kubernetes cluster with nodes in AWS EKS:
- ["eksctl"](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html)
- ["AWS Management Console and AWS CLI"](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html).

In this document, we'll introduce the "AWS Management Console and AWS CLI" method.

## Prerequisites

Before starting this tutorial, you must install and configure the following tools and resources that you need to create and manage an Amazon EKS cluster.

- AWS CLI – A command line tool for working with AWS services, including Amazon EKS. For more information, see ["Installing, updating, and uninstalling the AWS CLI"](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the AWS Command Line Interface User Guide. After installing the AWS CLI, we recommend that you also configure it. For more information, see ["Quick configuration with aws configure"](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html#cli-configure-quickstart-config) in the AWS Command Line Interface User Guide.

- kubectl – A command line tool for working with Kubernetes clusters. For more information, see ["Installing or updating kubectl"](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html).

- Required IAM permissions – The IAM security principal that you're using must have permissions to work with Amazon EKS IAM roles, service linked roles, AWS CloudFormation, a VPC, and related resources. For more information, see ["Actions, resources, and condition keys for Amazon Elastic Kubernetes Service"](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonelastickubernetesservice.html) and ["Using service-linked roles"](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html) in the IAM User Guide. You must complete all steps in this guide as the same user. To check the current user, run the following command:

```
aws sts get-caller-identity
```

## Create AWS EKS Cluster in AWS Console

You can refer to the YouTube video that demonstrates the steps to create an EKS cluster in the AWS console:
https://www.youtube.com/watch?v=KxxgF-DAGWc

Alternatively, you can refer to the AWS documentation directly: ["AWS Management Console and AWS CLI"](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-console.html)

## Uploading images to an AWS Private Registry

There are several reasons why your images might not be uploaded to a public image repository like Docker Hub.
You can upload your image to an AWS private registry using the following steps:

1. Create a new ECR repository (if not already created):

An Amazon ECR private repository contains your Docker images, Open Container Initiative (OCI) images, and OCI compatible artifacts. More information about Amazon ECR private repository: https://docs.aws.amazon.com/AmazonECR/latest/userguide/Repositories.html

```
aws ecr create-repository --repository-name my-app-repo --region <region>
```

Replace my-app-repo with your desired repository name and <region> with your AWS region (e.g., us-west-1).

2. Authenticate Docker to Your ECR Registry:

```
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <account_id>.dkr.ecr.<region>.amazonaws.com
```

Replace <region> with your AWS region and <account_id> with your AWS account ID.

3. Build Your Docker Image:

```
docker build -t my-app:<tag> .
```

4. Tag your Docker image so that it can be pushed to your ECR repository:

```
docker tag my-app:<tag> <account_id>.dkr.ecr.<region>.amazonaws.com/my-app-repo:<tag>
```

Replace <account_id> with your AWS account ID, <region> with your AWS region, and my-app-repo with your repository name.

5. Push your Docker image to the ECR repository with this command:

```
docker push <account_id>.dkr.ecr.<region>.amazonaws.com/my-app-repo:latest
```
Loading

0 comments on commit b381b2f

Please sign in to comment.