Secure Kubernets HA cluster in AWS using kube-aws

This repository contains an example of how to deploy a secure Kubernetes HA cluster in AWS using kube-aws automatically.

The fallowing setup use a base CloudFormation stack to configure Public and Private Subnets, IGW, NatGW, Route Tables, KMS and deploys automatically a VPN server in a public subnet. After the stack is created, the Kubernetes cluster is automatically deployed on top of it using kube-aws.

Features:

• simple and interactive deployment
• all the nodes are deployed in private subnets
• 3 distinct availability zones
• multi AZ masters
• workers configured using node pools, similar to GKE node pools
• HA ETCD with encrypted partitions for data, automatic backups to S3 and automatic/manual recovery from failover
• role based authentication using the RBAC plugin
• users authentication using OpenID Connect Identity (OIDC)
• AWS IAM roles directly assigned to pods using kube2iam
• cluster autoscaling
• VPN server automatically deployed to a public subnet

Deploy the Kubernetes cluster

1. Clone this repository locally

2. run ./deploy and fallow the instructions

3. Access your Kubernetes cluster. Since all the resources are in private networks, in order to access it, you'll need a VPN placed in one of the public subnets.Pritunl is now automatically deployed to a public subnet with a Elastic IP and DNS reccord.

Optionally you can configure your ~/.kube/configaccording to kubeconfig file to avoid passing the --kubeconfig flag on your commands.

Important

In order to expose public services using ELB or Ingress, the public subnets have to be tagged with the cluster name.

Ex. KubeernetesCluser=cluster_name

This is now set automatically

Add-ons

Note: all the addons can now be deployed automatically using addons/deploy script

Route53

This add-on is based on ExternalDNS project which allows you to control Route53 DNS records dynamically via Kubernetes resources.

Note: before deploying this addon, you have to create a IAM role and setup a trust relationship

Nginx Ingress Controller

Nginx ingress controller is deployed behind a ELB configured with Proxy Protocol. This way the ingress external address will be always associated with your ELB. Also you don't have to expose your workers publicly and get better protection from your ELB.

kube-lego

Kube-Lego automatically requests certificates for Kubernetes Ingress resources from Let's Encrypt.

fluentd-cloudwatch

This add-on is based on fluentd-kubernetes-daemonset and can forward the container logs to CloudWatchLogs.

Monitoring

A easy to setup, in-cluster, monitoring solution using Prometheus is available here