Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform template for different GenAI Examples #427

Open
arun-gupta opened this issue Jul 18, 2024 · 15 comments
Open

Terraform template for different GenAI Examples #427

arun-gupta opened this issue Jul 18, 2024 · 15 comments
Assignees
Labels
Dev enhancement New feature or request OPEAHack Issue created for OPEA Hackathon

Comments

@arun-gupta
Copy link
Contributor

There is a Terraform template for Chat QnA sample at https://github.com/intel/terraform-intel-aws-vm/tree/main/examples/gen-ai-xeon-opea-chatqna. This is only targeted at AWS.

We need a Terraform template for all GenAI examples that can be used to deploy on different cloud providers. Here are the target CSPs:

  • AWS
  • Azure
  • Google Cloud
  • Oracle Cloud

Each template should require the least amount of configuration, assume reasonable defaults yet configurable, and be able to deploy the entire sample with a public IP address to run the sample.

@tedivm
Copy link

tedivm commented Jul 18, 2024

What is the end result goal? Are we attempting to make it easy to create a demo, or do we want to make reusable modules that allow others to launch production environments? The reason I ask is that the approach is going to be very different depending on the goal.

I think ultimately we want to build reusable modules, but there is a lot that will go into that so I also understand why pushing out simple examples first would make sense.

@arun-gupta
Copy link
Contributor Author

The answer is "and" instead of "or". I'd like to have a simple demo that shows a quick run of OPEA on these CSPs. And then eventually modules that others can build on.

@mkbhanda
Copy link
Collaborator

We have a template for AWS, but looking to cover other CSPs and more of the GenAIExamples. https://github.com/intel/optimized-cloud-recipes/tree/main/recipes/ai-opea-chatqna-xeon

@lucasmelogithub
Copy link

lucasmelogithub commented Aug 29, 2024

Our Intel team has developed and maintained 31 Terraform Modules and some Ansible Modules.

TF Modules: https://github.com/orgs/intel/repositories?q=terraform-&type=all&language=&sort=
Ansible Modules: https://github.com/intel/optimized-cloud-recipes/tree/main/recipes

And as pointed out, we are now showcasing OPEA examples for end-to-end. @wsfowler kicked us off with the first one for AWS that was linked above.
We plan to add others but lean on OPEA components/examples. The stronger the OPEA examples are, the better these will be.
We are adding the glue between OPEA and cloud deployments, starting with single nodes.

The current focus is Azure/AWS/GCP, no plans yet to support other CSPs.

I'll work with the team to focus on two new ones:

  • Azure ChatQna RAG
  • GCP ChatQna RAG

Do we know which OPEA use cases(page views?) are getting the most attention so we can prioritize?

Does this help? Open to discussing if needed(MS Teams).

@lucasmelogithub
Copy link

By the way we keep them modular for many reasons.

  • The Terraform modules are used by customers/anyone, some include Intel Cloud optimizations built in. But the examples can be more specialized, like for OPEA.
  • The Ansible modules are used by customers/anyone running Ubuntu/Redhat, in the cloud or on-prem. That's the benefit of the modular approach, they don't depend on Terraform at all.
  • Now with OPEA examples, we can link it all together by leveraging all the awesome work the OPEA developers/community is providing.

They all work independently, but better together.
Legos, building blocks, solutions.

@arun-gupta
Copy link
Contributor Author

ChatQnA on Amazon, Azure and GCP are a great start.

Can the Terraform templates show how to deploy this sample on CSP's k8s service?

Are the Terraform templates modular where other samples can be accommodated when ready?

Are there any discussions about Ansible script to deploy on OpenShift?

@lucasmelogithub
Copy link

lucasmelogithub commented Aug 29, 2024

ChatQnA on Amazon, Azure and GCP are a great start.

Cool, thanks for the feedback.

Can the Terraform templates show how to deploy this sample on CSP's k8s service?

We have modules to deploy EKS, GKE, AKS. In short yes, we would depend and use OPEA helm charts as the integration point. If those are available, it should not be too hard.

Are the Terraform templates modular where other samples can be accommodated when ready?
Yes, since we have VM modules for AWS/Azure/GCP. All we need is to automate the OPEA deployments with Ansible Modules and then create new Terraform examples like we did for AWS ChatQnA.

For example, the same OPEA ChatQnA Ansible Module will be used on the Azure/GCP Terraform examples.

Are there any discussions about Ansible script to deploy on OpenShift?

We have not looked into OpenShift at all. Due to priorities, so far, the focus has been on first party cloud services.
Once we developed the other two, my focus was going to be on developing Ansible Modules for other use cases, ex: CodeGen, for more Xeon single node Terraform examples

@arun-gupta
Copy link
Contributor Author

I would suggest the following priority:

ChatQnA on Amazon, Microsoft, Azure
ChatQnA on OpenShift
ChatQnA on EKS, GKE, AKS

@lucasmelogithub
Copy link

I would suggest the following priority:

ChatQnA on Amazon, Microsoft, Azure ChatQnA on OpenShift ChatQnA on EKS, GKE, AKS

Thanks for the feedback, let me bring this up with the team.

@mkbhanda
Copy link
Collaborator

So @arun-gupta is suggesting first single node deployments on AWS and Microsoft Azure, then Red Hat OpenShift followed by Kubernetes variants on popular CSPs: Amazon, Google, and Microsoft Azure.

@arun-gupta
Copy link
Contributor Author

@mkbhanda just to be clear, single node deployments on AWS, Microsoft and Google using Docker Compose, then Red Hat Open Shift, and K8s distros on hyperscalers in the order mentioned above.

@lucasmelogithub
Copy link

@mkbhanda just to be clear, single node deployments on AWS, Microsoft and Google using Docker Compose, then Red Hat Open Shift, and K8s distros on hyperscalers in the order mentioned above.

Yes, for single nodes we will leverage the OPEA docker compose files.
For K8, we will hopefully use the OPEA helm charts.

@chickenrae
Copy link
Member

This is an issue for the OPEA Hackathon, if @lucasmelogithub you are going to do this during October great, if not, can you unassign yourself so we can have someone work on it?

@lucasmelogithub
Copy link

PR for ChatQnA for AWS and GCP created

#970

@poussa
Copy link

poussa commented Oct 21, 2024

PR for ChatQnA for AWS EKS including persistent volume support for model data and load balancer service type for external consumtion.

opea-project/GenAIInfra#480

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dev enhancement New feature or request OPEAHack Issue created for OPEA Hackathon
Projects
None yet
Development

No branches or pull requests

8 participants