EKS GitOps flow example configuration (#181)

* Add example TF configuration for GitOps onboarding flow

examples/eks/README.md

@@ -0,0 +1,10 @@
+## CAST AI EKS clusters examples
+
+- [Read only EKS cluster](eks_cluster_readonly/) - Onboard EKS cluster to CAST AI in read-only mode.
+- [Read-only EKS cluster with possible migration to Full access mode](eks_cluster_optional_readonly/) - Onboard EKS cluster to CAST AI in read-only mode with possibility to onboard to Full Access mode by controlling the variable `readonly=true`.
+- [EKS cluster](eks_cluster_assumerole/) - Onboard EKS cluster to CAST AI in Full Access mode. [CAST AI IAM module](https://github.com/castai/terraform-castai-eks-role-iam) is used to create required IAM resources.
+- [EKS cluster with custom IAM polices](eks_cluster_custom_iam/) -  Onboard EKS cluster to CAST AI in Full Access mode. Custom IAM policies are used to configure required IAM resources.
+- [EKS cluster with autoscaler policies](eks_cluster_autoscaler_policies/) -  Onboard EKS cluster to CAST AI in Full Access mode with configured autoscaler CAST AI policies.
+- [EKS cluster onboarded through GitOps](eks_cluster_gitops/) - Onboard EKS cluster to CAST AI in Full Access mode when CAST AI K8s components are installed using GitOps (e.g. ArgoCD, manual Helm releases). This example can also be used to import CAST AI cluster to Terraform if it was onboarded using [onboarding script](https://docs.cast.ai/docs/cluster-onboarding).
+- [EKS cluster with demo application](eks_cluster_webshop) - Onboard EKS cluster to CAST AI in Full Access mode. This example contains Demo application installed into cluster to showcase CAST AI cost saving capabilities.
+

examples/eks/eks_cluster_gitops/README.md

@@ -0,0 +1,27 @@
+## EKS and CAST AI example for GitOps onboarding flow
+
+Following example shows how to onboard EKS cluster to CAST AI using GitOps flow.
+In GitOps flow CAST AI Node Configuration, Node Templates and Autoscaler policies are managed using Terraform, but all Castware components such as `castai-agent`, `castai-cluster-controller`, `castai-evictor`, `castai-spot-handler`, `castai-kvisor` are to be installed using other means (e.g ArgoCD, manual Helm releases, etc.)
+
+Steps to take to successfully onboard EKS cluster to CAST AI using GitOps flow:
+Prerequisites:
+- CAST AI account
+- Obtained CAST AI [API Access key](https://docs.cast.ai/docs/authentication#obtaining-api-access-key) with Full Access
+1. Configure `tf.vars.example` file with required values. If EKS cluster is already managed by Terraform you could instead directly reference those resources.
+2. Run `terraform init`
+3. Run `terraform apply` and make a note of `cluster_id` and `cluster_token` output values. At this stage you would see that your cluster is in `Connecting` state in CAST AI console
+4. Install CAST AI components using Helm. Use `cluster_id` and `cluster_token` values to configure Helm releases:
+- Set `castai.apiKey` property to `cluster_token` for following CAST AI components: `castai-agent`, `castai-cluster-controller`, `castai-kvisor`
+- Set `additionalEnv.STATIC_CLUSTER_ID` property to `cluster_id` for`castai-agent` Helm release
+- Set 'castai.clusterID' property to for `castai-cluster-controller`, `castai-spot-handler`, `castai-kvisor`
+Example Helm install command:
+```helm install castai-cluster-controller cluster-controller --namespace=castai-agent --set castai.apiKey=<cluster_token>,provider=eks,castai.clusterID=<cluster_id>,createNamespace=false,apiURL="https://api.cast.ai"```
+5. Update [aws-auth](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) configmap with instance profile used by CAST AI. This instance profile is used by CAST AI managed nodes to communicate with EKS control plane.  Example of entry can be found [here](https://github.com/castai/terraform-provider-castai/blob/157babd57b0977f499eb162e9bee27bee51d292a/examples/eks/eks_cluster_assumerole/eks.tf#L28-L38).
+6. After all CAST AI components are installed in the cluster its status in CAST AI console would change from `Connecting` to `Connected` which means that cluster onboarding process completed successfully.
+
+
+## Importing already onboarded cluster to Terraform
+
+This example can also be used to import EKS cluster to Terraform which is already onboarded to CAST AI console trough [script](https://docs.cast.ai/docs/cluster-onboarding#how-it-works).   
+For importing existing cluster follow steps 1-3 above and change `castai_node_configuration.default` Node Configuration name.
+This would allow to manage already onboarded clusters' CAST AI Node Configurations and Node Templates through IaC.

examples/eks/eks_cluster_gitops/main.tf

@@ -0,0 +1,25 @@
+resource "castai_eks_cluster" "my_castai_cluster" {
+  account_id = var.aws_account_id
+  region     = var.aws_cluster_region
+  name       = var.aws_cluster_name
+
+  delete_nodes_on_disconnect = var.delete_nodes_on_disconnect
+  assume_role_arn            = var.aws_assume_role_arn
+}
+
+resource "castai_node_configuration" "default" {
+  cluster_id        = castai_eks_cluster.my_castai_cluster.id
+  name              = "default"
+  disk_cpu_ratio    = 0
+  min_disk_size     = 100
+  subnets           = var.subnets
+  eks {
+    security_groups      = var.security_groups
+    instance_profile_arn = var.instance_profile
+  }
+}
+
+resource "castai_node_configuration_default" "this" {
+  cluster_id       = castai_eks_cluster.my_castai_cluster.id
+  configuration_id = castai_node_configuration.default.id
+}

examples/eks/eks_cluster_gitops/outputs.tf

@@ -0,0 +1,10 @@
+output "cluster_id" {
+  value       = castai_eks_cluster.my_castai_cluster.id
+  description = "CAST AI cluster ID"
+}
+
+output "cluster_token" {
+  value       = castai_eks_cluster.my_castai_cluster.cluster_token
+  description = "CAST AI cluster token used by Castware to atuhenticate to Mothership"
+  sensitive   = true
+}

examples/eks/eks_cluster_gitops/providers.tf

@@ -0,0 +1,4 @@
+provider "castai" {
+  api_url   = var.castai_api_url
+  api_token = var.castai_api_token
+}

examples/eks/eks_cluster_gitops/tf.vars.example

@@ -0,0 +1,8 @@
+castai_api_token    = "PLACEHOLDER"
+aws_account_id      = "PLACEHOLDER"
+aws_cluster_region  = "PLACEHOLDER"
+aws_cluster_name    = "PLACEHOLDER"
+aws_assume_role_arn = "PLACEHOLDER"
+subnets             = ["PLACEHOLDER1", "PLACEHOLDER2"]
+security_groups     = ["PLACEHOLDER1", "PLACEHOLDER2"]
+instance_profile    = "PLACEHOLDER"

examples/eks/eks_cluster_gitops/variables.tf

@@ -0,0 +1,58 @@
+## Required variables.
+
+variable "aws_account_id" {
+  type        = string
+  description = "ID of AWS account the cluster is located in."
+}
+
+variable "aws_cluster_region" {
+  type        = string
+  description = "Region of the cluster to be connected to CAST AI."
+}
+
+variable "aws_cluster_name" {
+  type        = string
+  description = "Name of the cluster to be connected to CAST AI."
+}
+
+variable "castai_api_token" {
+  type = string
+  description = "CAST AI API token created in console.cast.ai API Access keys section"
+}
+
+variable "aws_assume_role_arn" {
+  type        = string
+  description = "Arn of the role to be used by CAST AI for IAM access"
+  default     = null
+}
+
+variable "subnets" {
+  type = list(string)
+  description = "Subnet IDs used by CAST AI to provision nodes"
+}
+
+variable "security_groups" {
+  type = list(string)
+  description = "Security Groups IDs used by CAST AI nodes to connect to K8s control plane, other nodes and have outbound access to Internet"
+}
+
+variable "instance_profile" {
+  type = string
+  description = "Instance profile ARN used by CAST AI provisioned nodes to connect to K8s control plane"
+}
+
+
+
+## Optional variables.
+
+variable "castai_api_url" {
+  type        = string
+  description = "CAST AI url to API, default value is https://api.cast.ai"
+  default     = "https://api.cast.ai"
+}
+
+variable "delete_nodes_on_disconnect" {
+  type        = bool
+  description = "Optionally delete Cast AI created nodes when the cluster is destroyed"
+  default     = false
+}

examples/eks/eks_cluster_gitops/versions.tf

@@ -0,0 +1,10 @@
+terraform {
+  required_version = ">= 0.13"
+
+  required_providers {
+    castai = {
+      source  = "castai/castai"
+      version = ">= 3.11.0"
+    }
+  }
+}