Terraform template for different GenAI Examples

[arun-gupta]

There is a Terraform template for Chat QnA sample at https://github.com/intel/terraform-intel-aws-vm/tree/main/examples/gen-ai-xeon-opea-chatqna. This is only targeted at AWS.

We need a Terraform template for all GenAI examples that can be used to deploy on different cloud providers. Here are the target CSPs:

• AWS
• Azure
• Google Cloud
• Oracle Cloud

Each template should require the least amount of configuration, assume reasonable defaults yet configurable, and be able to deploy the entire sample with a public IP address to run the sample.

[tedivm]

What is the end result goal? Are we attempting to make it easy to create a demo, or do we want to make reusable modules that allow others to launch production environments? The reason I ask is that the approach is going to be very different depending on the goal.

I think ultimately we want to build reusable modules, but there is a lot that will go into that so I also understand why pushing out simple examples first would make sense.

[arun-gupta]

The answer is "and" instead of "or". I'd like to have a simple demo that shows a quick run of OPEA on these CSPs. And then eventually modules that others can build on.

[mkbhanda]

We have a template for AWS, but looking to cover other CSPs and more of the GenAIExamples. https://github.com/intel/optimized-cloud-recipes/tree/main/recipes/ai-opea-chatqna-xeon

[lucasmelogithub]

Our Intel team has developed and maintained 31 Terraform Modules and some Ansible Modules.

TF Modules: https://github.com/orgs/intel/repositories?q=terraform-&type=all&language=&sort=
Ansible Modules: https://github.com/intel/optimized-cloud-recipes/tree/main/recipes

And as pointed out, we are now showcasing OPEA examples for end-to-end. @wsfowler kicked us off with the first one for AWS that was linked above.
We plan to add others but lean on OPEA components/examples. The stronger the OPEA examples are, the better these will be.
We are adding the glue between OPEA and cloud deployments, starting with single nodes.

The current focus is Azure/AWS/GCP, no plans yet to support other CSPs.

I'll work with the team to focus on two new ones:

• Azure ChatQna RAG
• GCP ChatQna RAG

Do we know which OPEA use cases(page views?) are getting the most attention so we can prioritize?

Does this help? Open to discussing if needed(MS Teams).

[lucasmelogithub]

By the way we keep them modular for many reasons.

• The Terraform modules are used by customers/anyone, some include Intel Cloud optimizations built in. But the examples can be more specialized, like for OPEA.
• The Ansible modules are used by customers/anyone running Ubuntu/Redhat, in the cloud or on-prem. That's the benefit of the modular approach, they don't depend on Terraform at all.
• Now with OPEA examples, we can link it all together by leveraging all the awesome work the OPEA developers/community is providing.

They all work independently, but better together.
Legos, building blocks, solutions.

[arun-gupta]

ChatQnA on Amazon, Azure and GCP are a great start.

Can the Terraform templates show how to deploy this sample on CSP's k8s service?

Are the Terraform templates modular where other samples can be accommodated when ready?

Are there any discussions about Ansible script to deploy on OpenShift?

[lucasmelogithub]

ChatQnA on Amazon, Azure and GCP are a great start.

Cool, thanks for the feedback.

Can the Terraform templates show how to deploy this sample on CSP's k8s service?

We have modules to deploy EKS, GKE, AKS. In short yes, we would depend and use OPEA helm charts as the integration point. If those are available, it should not be too hard.

Are the Terraform templates modular where other samples can be accommodated when ready?
Yes, since we have VM modules for AWS/Azure/GCP. All we need is to automate the OPEA deployments with Ansible Modules and then create new Terraform examples like we did for AWS ChatQnA.

For example, the same OPEA ChatQnA Ansible Module will be used on the Azure/GCP Terraform examples.

Are there any discussions about Ansible script to deploy on OpenShift?

We have not looked into OpenShift at all. Due to priorities, so far, the focus has been on first party cloud services.
Once we developed the other two, my focus was going to be on developing Ansible Modules for other use cases, ex: CodeGen, for more Xeon single node Terraform examples

[arun-gupta]

I would suggest the following priority:

ChatQnA on Amazon, Microsoft, Azure
ChatQnA on OpenShift
ChatQnA on EKS, GKE, AKS

[lucasmelogithub]

I would suggest the following priority:

ChatQnA on Amazon, Microsoft, Azure ChatQnA on OpenShift ChatQnA on EKS, GKE, AKS

Thanks for the feedback, let me bring this up with the team.

[mkbhanda]

So @arun-gupta is suggesting first single node deployments on AWS and Microsoft Azure, then Red Hat OpenShift followed by Kubernetes variants on popular CSPs: Amazon, Google, and Microsoft Azure.

[arun-gupta]

@mkbhanda just to be clear, single node deployments on AWS, Microsoft and Google using Docker Compose, then Red Hat Open Shift, and K8s distros on hyperscalers in the order mentioned above.

[lucasmelogithub]

@mkbhanda just to be clear, single node deployments on AWS, Microsoft and Google using Docker Compose, then Red Hat Open Shift, and K8s distros on hyperscalers in the order mentioned above.

Yes, for single nodes we will leverage the OPEA docker compose files.
For K8, we will hopefully use the OPEA helm charts.

[chickenrae]

This is an issue for the OPEA Hackathon, if @lucasmelogithub you are going to do this during October great, if not, can you unassign yourself so we can have someone work on it?

[lucasmelogithub]

PR for ChatQnA for AWS and GCP created

#970

[poussa]

PR for ChatQnA for AWS EKS including persistent volume support for model data and load balancer service type for external consumtion.

opea-project/GenAIInfra#480