-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add beta cluster module with NodeLocal DNSCache capability (#33)
- Loading branch information
Luke Reed
authored
Mar 29, 2021
1 parent
8fe5851
commit 959f9dc
Showing
12 changed files
with
337 additions
and
25 deletions.
There are no files selected for viewing
Validating CODEOWNERS rules …
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
* @dosullivan | ||
* @lucasreed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
.terraform |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
## 1.4.1 | ||
### Initial Release | ||
* GKE Module that supports private and public cluster settings with beta features using the `google-beta` provider. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Terraform VPC Native GKE Cluster Module | ||
|
||
Requirements: | ||
`terraform` >= `0.12.x` | ||
terraform provider `google` >= `2.5.0` | ||
|
||
This module manages a Google Kubernetes Engine (GKE) VPC Native cluster. The subnet CIDRs used for cluster nodes, pods, and services, are specified in the form of existing Google Compute secondary IP ranges. Use a separate Terraform module, such as [this `terraform-gcp-vpc-native/default` one](https://github.com/FairwindsOps/terraform-gcp-vpc-native/tree/master/default), to create these network resources in advance. | ||
|
||
To provision a cluster with a private kubernetes api or a private node group, specify `enable_private_endpoint` and/or `enable_private_nodes`. If either private booleans are true, `master_ipv4_cidr_block` MUST be declared. | ||
|
||
For example: To have a Public API and private node groups, set `enable_private_nodes` to `true` and `master_ipv4_cidr_block` to a `x.x.x.x/28` IP range. | ||
|
||
See the file [example-usage](./example-usage) for an example of how to use this module. Below are the available module inputs: | ||
| Parameter | Description | Default | | ||
|------------------------------------|-----------------------------------------------------|-----------------------------------------| | ||
| `region` | GKE region | `None` | | ||
| `name` | Name of the GKE cluster | `None` | | ||
| `project` | GCP Project | `""` | | ||
| `network_name` | Existing google_compute_network | `None` | | ||
| `nodes_subnetwork_name` | Existing google_compute_subnetwork | `None` | | ||
| `kubernetes_version` | minimum version of master nodes | `None` | | ||
| `pods_secondary_ip_range_name` | IP range to be used for pods | `None` | | ||
| `services_secondary_ip_range_name` | IP range to be used for services | `None` | | ||
| `master_authorized_network_cidrs` | List of maps with authorized cidrs and descriptions | `see inputs.tf` | | ||
| `maintenance_policy_start_time` | Maintenance Window (GMT) | `06:00` | | ||
| `enable_private_endpoint` | Private Kube API endpoint | `false` | | ||
| `enable_private_nodes` | Private compute instances | `false` | | ||
| `enable_workload_identity` | Enable workload identity for the cluster | `false` | | ||
| `master_ipv4_cidr_block` | IPV4 CIDR block for controlplane (must be /28) | `null` | | ||
| `monitoring_service` | The monitoring service to write metrics to | `monitoring.googleapis.com/kubernetes` | | ||
| `logging_service` | The logging service to write logs to | `logging.googleapis.com/kubernetes` | | ||
| `vpa_enabled` | A boolean to enable VPA for the cluster | `false` | | ||
| `enable_workload_identity` | A boolean to enable workload identity | `false` | | ||
| `enable_shielded_nodes` | A boolean to enable cluster-wide shielded nodes | `false` | | ||
| `enable_node_local_dns_cache` | A boolean to enable NodeLocal DNSCache | `true` | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# These local variables can be used as inputs to both a network and this GKE VPC Native cluster module. | ||
locals { | ||
region = "us-central1" | ||
network_name = "customername" | ||
kubernetes_version = "1.13.9-gke.3" | ||
master_ipv4_cidr_block = "10.128.254.0/28" | ||
} | ||
|
||
|
||
# The `module.customername_vpc` below refers to an instance of a VPC module. | ||
# Ref: https://github.com/FairwindsOps/terraform-gcp-vpc-native | ||
module "customername_cluster" { | ||
# Change the ref below to use a vX.Y.Z release instead of master. | ||
source = "git@github.com:/FairwindsOps/terraform-gke//vpc-native?ref=master" | ||
|
||
name = "customername-cluster1" | ||
region = local.region | ||
project = "customername-dev" | ||
kubernetes_version = local.kubernetes_version | ||
network_name = local.network_name | ||
nodes_subnetwork_name = module.customername_vpc.subnetwork | ||
pods_secondary_ip_range_name = module.customername_vpc.gke_pods_1 | ||
services_secondary_ip_range_name = module.customername_vpc.gke_services_1 | ||
# private cluster options | ||
enable_private_endpoint = false | ||
enable_private_nodes = true | ||
master_ipv4_cidr_block = local.master_ipv4_cidr_block | ||
|
||
master_authorized_network_cidrs = [ | ||
{ | ||
# This is the module default, but demonstrates specifying this input. | ||
cidr_block = "0.0.0.0/0" | ||
display_name = "from the Internet" | ||
}, | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,98 @@ | ||
variable "name" { | ||
description = "The name of the GKE cluster" | ||
} | ||
|
||
variable "region" { | ||
description = "The region where the GKE cluster will be created." | ||
} | ||
|
||
variable "project" { | ||
description = "The project where the GKE cluster will be created. Leave unspecified to use the project from the provider." | ||
default = "" | ||
} | ||
|
||
variable "network_name" { | ||
description = "The name of an existing google_compute_network resource to which the cluster will be connected." | ||
} | ||
|
||
variable "nodes_subnetwork_name" { | ||
description = "The name of an existing google_compute_subnetwork resource where cluster compute instances are launched." | ||
} | ||
|
||
variable "kubernetes_version" { | ||
description = "The minimum version of master nodes. This can be changed to upgrade the cluster - remember to upgrade the Kubernetes version for node pools (managed separately)." | ||
} | ||
|
||
variable "pods_secondary_ip_range_name" { | ||
description = "The name of an existing network secondary IP range to be used for pods." | ||
} | ||
|
||
variable "services_secondary_ip_range_name" { | ||
description = "The name of an existing network secondary IP range to be used for services." | ||
} | ||
|
||
variable "master_authorized_network_cidrs" { | ||
type = list | ||
description = "A list of up to 20 maps containing `master_authorized_network_cidrs` and `display_name` keys, representing source network CIDRs that are allowed to connect master nodes over HTTPS." | ||
|
||
default = [ | ||
{ | ||
## this needs to be changed when enable_private_endpoint is true | ||
cidr_block = "0.0.0.0/0" | ||
display_name = "everywhere" | ||
}, | ||
] | ||
} | ||
|
||
variable "maintenance_policy_start_time" { | ||
description = "The time (in GMT) when the cluster maintenance window will start." | ||
default = "06:00" | ||
} | ||
|
||
variable "enable_private_endpoint" { | ||
description = "A boolean to enable private (non public) kube-api endpoints" | ||
default = false | ||
} | ||
|
||
variable "enable_private_nodes" { | ||
description = "A boolean to enable private (non public) nodes" | ||
default = false | ||
} | ||
|
||
variable "master_ipv4_cidr_block" { | ||
description = "The /28 range for the master instances. Must be set if enable_private_nodes or enable_private_endpoint is true" | ||
default = null | ||
} | ||
|
||
variable "monitoring_service" { | ||
description = "The monitoring service to write metrics to" | ||
default = "monitoring.googleapis.com/kubernetes" | ||
} | ||
|
||
variable "logging_service" { | ||
description = "The logging service to write logs to" | ||
default = "logging.googleapis.com/kubernetes" | ||
} | ||
|
||
variable "vpa_enabled" { | ||
description = "A boolean to enable VPA for the cluster" | ||
default = false | ||
} | ||
|
||
variable "enable_workload_identity" { | ||
type = bool | ||
description = "A boolean to enable workload identity" | ||
default = false | ||
} | ||
|
||
variable "enable_shielded_nodes" { | ||
type = bool | ||
description = "A boolean to enable cluster-wide shielded nodes" | ||
default = false | ||
} | ||
|
||
variable "enable_node_local_dns_cache" { | ||
type = bool | ||
description = "A boolean to enable NodeLocal DNSCache" | ||
default = true | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
locals { | ||
cluster_workload_identity_namespace = var.enable_workload_identity ? ["${var.project}.svc.id.goog"] : [] | ||
} | ||
|
||
resource "google_container_cluster" "cluster" { | ||
provider = google-beta | ||
name = var.name | ||
location = var.region | ||
min_master_version = var.kubernetes_version | ||
network = var.network_name | ||
subnetwork = var.nodes_subnetwork_name | ||
monitoring_service = var.monitoring_service | ||
logging_service = var.logging_service | ||
enable_shielded_nodes = var.enable_shielded_nodes | ||
|
||
|
||
vertical_pod_autoscaling { | ||
enabled = var.vpa_enabled | ||
} | ||
|
||
ip_allocation_policy { | ||
cluster_secondary_range_name = var.pods_secondary_ip_range_name | ||
services_secondary_range_name = var.services_secondary_ip_range_name | ||
} | ||
|
||
# This is believed to apply to the default node pool, which gets created then deleted. | ||
initial_node_count = 1 | ||
remove_default_node_pool = true | ||
|
||
# The absence of a user and password here disables basic auth | ||
master_auth { | ||
username = "" | ||
password = "" | ||
|
||
client_certificate_config { | ||
issue_client_certificate = false | ||
} | ||
} | ||
|
||
private_cluster_config { | ||
enable_private_endpoint = var.enable_private_endpoint | ||
enable_private_nodes = var.enable_private_nodes | ||
master_ipv4_cidr_block = var.master_ipv4_cidr_block | ||
} | ||
|
||
addons_config { | ||
network_policy_config { | ||
disabled = false | ||
} | ||
dns_cache_config { | ||
enabled = var.enable_node_local_dns_cache | ||
} | ||
} | ||
|
||
dynamic "workload_identity_config" { | ||
for_each = local.cluster_workload_identity_namespace | ||
content { | ||
identity_namespace = local.cluster_workload_identity_namespace[0] | ||
} | ||
} | ||
|
||
network_policy { | ||
enabled = true | ||
} | ||
|
||
master_authorized_networks_config { | ||
dynamic "cidr_blocks" { | ||
for_each = var.master_authorized_network_cidrs | ||
content { | ||
# TF-UPGRADE-TODO: The automatic upgrade tool can't predict | ||
# which keys might be set in maps assigned here, so it has | ||
# produced a comprehensive set here. Consider simplifying | ||
# this after confirming which keys can be set in practice. | ||
|
||
cidr_block = cidr_blocks.value.cidr_block | ||
display_name = lookup(cidr_blocks.value, "display_name", null) | ||
} | ||
} | ||
} | ||
|
||
maintenance_policy { | ||
daily_maintenance_window { | ||
start_time = var.maintenance_policy_start_time | ||
} | ||
} | ||
|
||
resource_labels = { | ||
kubernetescluster = var.name | ||
} | ||
|
||
lifecycle { | ||
# ignore changes to node_pool specifically so it doesn't | ||
# try to recreate default node pool with every change | ||
# ignore changes to network and subnetwork so it doesn't | ||
# clutter up diff with dumb changes like: | ||
# projects/[name]/regions/us-central1/subnetworks/[name]" => "name" | ||
ignore_changes = [ | ||
node_pool, | ||
network, | ||
subnetwork, | ||
] | ||
} | ||
} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
output "name" { | ||
description = "The static name of the GKE cluster" | ||
value = google_container_cluster.cluster.name | ||
} | ||
|
||
output "endpoint" { | ||
description = "The GKE Cluster Endpoints IP" | ||
value = google_container_cluster.cluster.endpoint | ||
} | ||
|
||
## This is passed back out in case it's needed to inherit for node pools | ||
output "kubernetes_version" { | ||
description = "The Kubernetes version used when creating or upgrading this cluster. This does not reflect the current version of master or worker nodes." | ||
value = var.kubernetes_version | ||
} | ||
|
||
output "master_version" { | ||
description = "The current version of the Kubernetes master nodes, which will differ from the kubernetes_version output if GKE upgrades masters automatically." | ||
value = google_container_cluster.cluster.master_version | ||
} | ||
|
||
output "region" { | ||
description = "The region in which this cluster exists" | ||
value = var.region | ||
} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
|
||
terraform { | ||
required_version = ">= 0.12" | ||
required_providers { | ||
google-beta = ">=2.5.0" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,5 @@ | ||
|
||
# Changelog | ||
All notable changes to this project will be documented in this file. | ||
|
||
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) | ||
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html). | ||
|
||
## vpc-native-v1.4.1 | ||
* Added the ability to use shielded nodes in a cluster | ||
## 1.0.0 | ||
### Initial Release | ||
* GKE Module that supports private and public cluster settings | ||
* GKE Module that supports private and public cluster settings |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters