Add beta cluster module with NodeLocal DNSCache capability (#33)

CODEOWNERS

@@ -1 +1 @@
-* @dosullivan
+* @lucasreed

node_pool/CHANGELOG.md

@@ -1,3 +1,5 @@
+# node-pool-v3.3.1
+- Added the ability to enable secure boot on nodes when the cluster uses shielded nodes. This can be enabled with the variable `enable_secure_boot`
 # node-pool-v3.3.0
 - Added `node_metadata` parameter to control node metadata provided to workload, so that [workload identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) functionality may be used.
 

vpc-native-beta/.gitignore

@@ -0,0 +1 @@
+.terraform

vpc-native-beta/CHANGELOG.md

@@ -0,0 +1,3 @@
+## 1.4.1
+### Initial Release
+* GKE Module that supports private and public cluster settings with beta features using the `google-beta` provider.

vpc-native-beta/README.md

@@ -0,0 +1,35 @@
+# Terraform VPC Native GKE Cluster Module
+
+Requirements:
+`terraform` >= `0.12.x`
+terraform provider `google` >= `2.5.0`
+
+This module manages a Google Kubernetes Engine (GKE) VPC Native cluster. The subnet CIDRs used for cluster nodes, pods, and services, are specified in the form of existing Google Compute secondary IP ranges. Use a separate Terraform module, such as [this `terraform-gcp-vpc-native/default` one](https://github.com/FairwindsOps/terraform-gcp-vpc-native/tree/master/default), to create these network resources in advance.
+
+To provision a cluster with a private kubernetes api or a private node group, specify `enable_private_endpoint` and/or `enable_private_nodes`. If either private booleans are true, `master_ipv4_cidr_block` MUST be declared.
+
+For example: To have a Public API and private node groups, set `enable_private_nodes` to `true` and `master_ipv4_cidr_block` to a `x.x.x.x/28` IP range.
+
+See the file [example-usage](./example-usage) for an example of how to use this module. Below are the available module inputs:
+| Parameter                          | Description                                         | Default                                 |
+|------------------------------------|-----------------------------------------------------|-----------------------------------------|
+| `region`                           | GKE region                                          | `None`                                  |
+| `name`                             | Name of the GKE cluster                             | `None`                                  |
+| `project`                          | GCP Project                                         | `""`                                    |
+| `network_name`                     | Existing google_compute_network                     | `None`                                  |
+| `nodes_subnetwork_name`            | Existing google_compute_subnetwork                  | `None`                                  |
+| `kubernetes_version`               | minimum version of master nodes                     | `None`                                  |
+| `pods_secondary_ip_range_name`     | IP range to be used for pods                        | `None`                                  |
+| `services_secondary_ip_range_name` | IP range to be used for services                    | `None`                                  |
+| `master_authorized_network_cidrs`  | List of maps with authorized cidrs and descriptions | `see inputs.tf`                         |
+| `maintenance_policy_start_time`    | Maintenance Window (GMT)                            | `06:00`                                 |
+| `enable_private_endpoint`          | Private Kube API endpoint                           | `false`                                 |
+| `enable_private_nodes`             | Private compute instances                           | `false`                                 |
+| `enable_workload_identity`         | Enable workload identity for the cluster            | `false`                                 |
+| `master_ipv4_cidr_block`           | IPV4 CIDR block for controlplane (must be /28)      | `null`                                  |
+| `monitoring_service`               | The monitoring service to write metrics to          | `monitoring.googleapis.com/kubernetes`  |
+| `logging_service`                  | The logging service to write logs to                | `logging.googleapis.com/kubernetes`     |
+| `vpa_enabled`                      | A boolean to enable VPA for the cluster             | `false`                                 |
+| `enable_workload_identity`         | A boolean to enable workload identity               | `false`                                 |
+| `enable_shielded_nodes`            | A boolean to enable cluster-wide shielded nodes     | `false`                                 |
+| `enable_node_local_dns_cache`      | A boolean to enable NodeLocal DNSCache              | `true`                                  |

vpc-native-beta/example-usage

@@ -0,0 +1,36 @@
+# These local variables can be used as inputs to both a network and this GKE VPC Native cluster module.
+locals {
+  region                 = "us-central1"
+  network_name           = "customername"
+  kubernetes_version     = "1.13.9-gke.3"
+  master_ipv4_cidr_block = "10.128.254.0/28"
+}
+
+
+# The `module.customername_vpc` below refers to an instance of a VPC module.
+# Ref: https://github.com/FairwindsOps/terraform-gcp-vpc-native
+module "customername_cluster" {
+  # Change the ref below to use a vX.Y.Z release instead of master.
+  source = "git@github.com:/FairwindsOps/terraform-gke//vpc-native?ref=master"
+
+  name                             = "customername-cluster1"
+  region                           = local.region
+  project                          = "customername-dev"
+  kubernetes_version               = local.kubernetes_version
+  network_name                     = local.network_name
+  nodes_subnetwork_name            = module.customername_vpc.subnetwork
+  pods_secondary_ip_range_name     = module.customername_vpc.gke_pods_1
+  services_secondary_ip_range_name = module.customername_vpc.gke_services_1
+  # private cluster options
+  enable_private_endpoint = false
+  enable_private_nodes    = true
+  master_ipv4_cidr_block  = local.master_ipv4_cidr_block
+
+  master_authorized_network_cidrs = [
+    {
+      # This is the module default, but demonstrates specifying this input.
+      cidr_block   = "0.0.0.0/0"
+      display_name = "from the Internet"
+    },
+  ]
+}

vpc-native-beta/inputs.tf

@@ -0,0 +1,98 @@
+variable "name" {
+  description = "The name of the GKE cluster"
+}
+
+variable "region" {
+  description = "The region where the GKE cluster will be created."
+}
+
+variable "project" {
+  description = "The project where the GKE cluster will be created. Leave unspecified to use the project from the provider."
+  default     = ""
+}
+
+variable "network_name" {
+  description = "The name of an existing google_compute_network resource to which the cluster will be connected."
+}
+
+variable "nodes_subnetwork_name" {
+  description = "The name of an existing google_compute_subnetwork resource where cluster compute instances are launched."
+}
+
+variable "kubernetes_version" {
+  description = "The minimum version of master nodes. This can be changed to upgrade the cluster - remember to upgrade the Kubernetes version for node pools (managed separately)."
+}
+
+variable "pods_secondary_ip_range_name" {
+  description = "The name of an existing network secondary IP range to be used for pods."
+}
+
+variable "services_secondary_ip_range_name" {
+  description = "The name of an existing network secondary IP range to be used for services."
+}
+
+variable "master_authorized_network_cidrs" {
+  type        = list
+  description = "A list of up to 20 maps containing `master_authorized_network_cidrs` and `display_name` keys, representing source network CIDRs that are allowed to connect master nodes over HTTPS."
+
+  default = [
+    {
+      ## this needs to be changed when enable_private_endpoint is true
+      cidr_block   = "0.0.0.0/0"
+      display_name = "everywhere"
+    },
+  ]
+}
+
+variable "maintenance_policy_start_time" {
+  description = "The time (in GMT) when the cluster maintenance window will start."
+  default     = "06:00"
+}
+
+variable "enable_private_endpoint" {
+  description = "A boolean to enable private (non public) kube-api endpoints"
+  default     = false
+}
+
+variable "enable_private_nodes" {
+  description = "A boolean to enable private (non public) nodes"
+  default     = false
+}
+
+variable "master_ipv4_cidr_block" {
+  description = "The /28 range for the master instances. Must be set if enable_private_nodes or enable_private_endpoint is true"
+  default     = null
+}
+
+variable "monitoring_service" {
+  description = "The monitoring service to write metrics to"
+  default     = "monitoring.googleapis.com/kubernetes"
+}
+
+variable "logging_service" {
+  description = "The logging service to write logs to"
+  default     = "logging.googleapis.com/kubernetes"
+}
+
+variable "vpa_enabled" {
+  description = "A boolean to enable VPA for the cluster"
+  default     = false
+}
+
+variable "enable_workload_identity" {
+  type        = bool
+  description = "A boolean to enable workload identity"
+  default     = false
+}
+
+variable "enable_shielded_nodes" {
+  type        = bool
+  description = "A boolean to enable cluster-wide shielded nodes"
+  default     = false
+}
+
+variable "enable_node_local_dns_cache" {
+  type        = bool
+  description = "A boolean to enable NodeLocal DNSCache"
+  default     = true
+}

vpc-native-beta/main.tf

@@ -0,0 +1,104 @@
+locals {
+  cluster_workload_identity_namespace = var.enable_workload_identity ? ["${var.project}.svc.id.goog"] : []
+}
+
+resource "google_container_cluster" "cluster" {
+  provider              = google-beta
+  name                  = var.name
+  location              = var.region
+  min_master_version    = var.kubernetes_version
+  network               = var.network_name
+  subnetwork            = var.nodes_subnetwork_name
+  monitoring_service    = var.monitoring_service
+  logging_service       = var.logging_service
+  enable_shielded_nodes = var.enable_shielded_nodes
+
+
+  vertical_pod_autoscaling {
+    enabled = var.vpa_enabled
+  }
+
+  ip_allocation_policy {
+    cluster_secondary_range_name  = var.pods_secondary_ip_range_name
+    services_secondary_range_name = var.services_secondary_ip_range_name
+  }
+
+  # This is believed to apply to the default node pool, which gets created then deleted.
+  initial_node_count       = 1
+  remove_default_node_pool = true
+
+  # The absence of a user and password here disables basic auth
+  master_auth {
+    username = ""
+    password = ""
+
+    client_certificate_config {
+      issue_client_certificate = false
+    }
+  }
+
+  private_cluster_config {
+    enable_private_endpoint = var.enable_private_endpoint
+    enable_private_nodes    = var.enable_private_nodes
+    master_ipv4_cidr_block  = var.master_ipv4_cidr_block
+  }
+
+  addons_config {
+    network_policy_config {
+      disabled = false
+    }
+    dns_cache_config {
+      enabled = var.enable_node_local_dns_cache
+    }
+  }
+
+  dynamic "workload_identity_config" {
+    for_each = local.cluster_workload_identity_namespace
+    content {
+      identity_namespace = local.cluster_workload_identity_namespace[0]
+    }
+  }
+
+  network_policy {
+    enabled = true
+  }
+
+  master_authorized_networks_config {
+    dynamic "cidr_blocks" {
+      for_each = var.master_authorized_network_cidrs
+      content {
+        # TF-UPGRADE-TODO: The automatic upgrade tool can't predict
+        # which keys might be set in maps assigned here, so it has
+        # produced a comprehensive set here. Consider simplifying
+        # this after confirming which keys can be set in practice.
+
+        cidr_block   = cidr_blocks.value.cidr_block
+        display_name = lookup(cidr_blocks.value, "display_name", null)
+      }
+    }
+  }
+
+  maintenance_policy {
+    daily_maintenance_window {
+      start_time = var.maintenance_policy_start_time
+    }
+  }
+
+  resource_labels = {
+    kubernetescluster = var.name
+  }
+
+  lifecycle {
+    # ignore changes to node_pool specifically so it doesn't
+    #   try to recreate default node pool with every change
+    # ignore changes to network and subnetwork so it doesn't
+    #   clutter up diff with dumb changes like:
+    #   projects/[name]/regions/us-central1/subnetworks/[name]" => "name"
+    ignore_changes = [
+      node_pool,
+      network,
+      subnetwork,
+    ]
+  }
+}
+

vpc-native-beta/outputs.tf

@@ -0,0 +1,26 @@
+output "name" {
+  description = "The static name of the GKE cluster"
+  value       = google_container_cluster.cluster.name
+}
+
+output "endpoint" {
+  description = "The GKE Cluster Endpoints IP"
+  value       = google_container_cluster.cluster.endpoint
+}
+
+## This is passed back out in case it's needed to inherit for node pools
+output "kubernetes_version" {
+  description = "The Kubernetes version used when creating or upgrading this cluster. This does not reflect the current version of master or worker nodes."
+  value       = var.kubernetes_version
+}
+
+output "master_version" {
+  description = "The current version of the Kubernetes master nodes, which will differ from the kubernetes_version output if GKE upgrades masters automatically."
+  value       = google_container_cluster.cluster.master_version
+}
+
+output "region" {
+  description = "The region in which this cluster exists"
+  value       = var.region
+}
+

vpc-native-beta/versions.tf

@@ -0,0 +1,7 @@
+
+terraform {
+  required_version = ">= 0.12"
+  required_providers {
+    google-beta = ">=2.5.0"
+  }
+}

vpc-native/CHANGELOG.md

@@ -1,10 +1,5 @@
-
-# Changelog
-All notable changes to this project will be documented in this file.
-
-The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
-and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
-
+## vpc-native-v1.4.1
+* Added the ability to use shielded nodes in a cluster
 ## 1.0.0
 ### Initial Release
-* GKE Module that supports private and public cluster settings
+* GKE Module that supports private and public cluster settings

vpc-native/README.md

@@ -11,19 +11,24 @@ To provision a cluster with a private kubernetes api or a private node group, sp
 For example: To have a Public API and private node groups, set `enable_private_nodes` to `true` and `master_ipv4_cidr_block` to a `x.x.x.x/28` IP range.
 
 See the file [example-usage](./example-usage) for an example of how to use this module. Below are the available module inputs:
-| Parameter                          | Description                                         | Default         |
-|------------------------------------|-----------------------------------------------------|-----------------|
-| `region`                           | GKE region                                          | `None`          |
-| `name`                             | Name of the GKE cluster                             | `None`          |
-| `project`                          | GCP Project                                         | `""`            |
-| `network_name`                     | Existing google_compute_network                     | `None`          |
-| `nodes_subnetwork_name`            | Existing google_compute_subnetwork                  | `None`          |
-| `kubernetes_version`               | minimum version of master nodes                     | `None`          |
-| `pods_secondary_ip_range_name`     | IP range to be used for pods                        | `None`          |
-| `services_secondary_ip_range_name` | IP range to be used for services                    | `None`          |
-| `master_authorized_network_cidrs`  | List of maps with authorized cidrs and descriptions | `see inputs.tf` |
-| `maintenance_policy_start_time`    | Maintenance Window (GMT)                            | `06:00`         |
-| `enable_private_endpoint`          | Private Kube API endpoint                           | `false`          |
-| `enable_private_nodes`             | Private compute instances                           | `false`          |
-| `enable_workload_identity`         | Enable workload identity for the cluster            | `false`         |
-| `master_ipv4_cidr_block`           | IPV4 CIDR block for controlplane (must be /28)      | `null`          |
+| Parameter                          | Description                                         | Default                                 |
+|------------------------------------|-----------------------------------------------------|-----------------------------------------|
+| `region`                           | GKE region                                          | `None`                                  |
+| `name`                             | Name of the GKE cluster                             | `None`                                  |
+| `project`                          | GCP Project                                         | `""`                                    |
+| `network_name`                     | Existing google_compute_network                     | `None`                                  |
+| `nodes_subnetwork_name`            | Existing google_compute_subnetwork                  | `None`                                  |
+| `kubernetes_version`               | minimum version of master nodes                     | `None`                                  |
+| `pods_secondary_ip_range_name`     | IP range to be used for pods                        | `None`                                  |
+| `services_secondary_ip_range_name` | IP range to be used for services                    | `None`                                  |
+| `master_authorized_network_cidrs`  | List of maps with authorized cidrs and descriptions | `see inputs.tf`                         |
+| `maintenance_policy_start_time`    | Maintenance Window (GMT)                            | `06:00`                                 |
+| `enable_private_endpoint`          | Private Kube API endpoint                           | `false`                                 |
+| `enable_private_nodes`             | Private compute instances                           | `false`                                 |
+| `enable_workload_identity`         | Enable workload identity for the cluster            | `false`                                 |
+| `master_ipv4_cidr_block`           | IPV4 CIDR block for controlplane (must be /28)      | `null`                                  |
+| `monitoring_service`               | The monitoring service to write metrics to          | `monitoring.googleapis.com/kubernetes`  |
+| `logging_service`                  | The logging service to write logs to                | `logging.googleapis.com/kubernetes`     |
+| `vpa_enabled`                      | A boolean to enable VPA for the cluster             | `false`                                 |
+| `enable_workload_identity`         | A boolean to enable workload identity               | `false`                                 |
+| `enable_shielded_nodes`            | A boolean to enable cluster-wide shielded nodes     | `false`                                 |