Terraform: EKS 구성

Terraform의 AWS 모듈을 활용해서 EKS를 구성해 보도록 하겠습니다.

수작업을 Code로 완전히 자동화 합니다.

Human Error 방지와 멱등성을 완벽하게 보장하며

그리인해서 경제적인 이득까지 얻을 수 있으니 ㅎㅎ

 

총 10개의 Module로 분리해 놓았습니다.

1. init

2. Region

3. VPC

4. Internet Gateway

5. Subnet

6. Routing Table

7. Routing Table Association

8. Security Group

9. EKS Cluster

10. EKS WorkNode 생성

순서 입니다.

 

 

1. init

 1.1. Terraform AWS 버젼 지정, 현재 최신버전은 5.7.0

terraform {
  required_providers {
    aws = {
        source = "hashicorp/aws"
        version = "~> 5.0" 
    }
  }
}

 

2. region

 2.1. region 지정 ap-northeast.2

 2.2. 변수화 시켜 나중 재사용 가능한 모듈작업 가능케 함.

provider "aws" {
  region = var.region
}

 

3. VPC

 3.1. VPC 생성

 3.2. tag의 Name은 본인 이니셜: var.name -> sdkim

resource "aws_vpc" "eks_vpc" {
  cidr_block           = var.cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name = "${var.name}-vpc"
  }
}

 

4. Internet Gateway

 4.1. Internet Gateway를 생성, 이를 이전 생성한 VPC 연결

resource "aws_internet_gateway" "eks_ig" {
  vpc_id = aws_vpc.eks_vpc.id

  tags = {
    Name = "${var.name}-ig"
  }
}

 

5. Subnet

 5.1. 4개의  Subnet 생성

 5.2. 2개는 Public, 2개는 Private으로

 5.3. 단 모두 map_public_ip_on_launch 라는 agrument를 사용해서 인스턴스 생성 시 자동으로 Public IP 부여

resource "aws_subnet" "eksnet_ma" {
  count                   = 2
  vpc_id                  = aws_vpc.eks_vpc.id
  cidr_block              = "10.0.${count.index}.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "c"}"

  tags = {
    Name = "${var.name}-pub-${count.index == 0 ? "a" : "c"}"
  }
}

resource "aws_subnet" "eksnet_work" {
  count                   = 2
  vpc_id                  = aws_vpc.eks_vpc.id
  cidr_block              = "10.0.${count.index + 2}.0/24"
  map_public_ip_on_launch = true
  availability_zone       = "${var.region}${count.index == 0 ? "a" : "c"}"

  tags = {
    Name = "${var.name}-pri-${count.index == 0 ? "a" : "c"}"
  }
}

 

6. Routing Table

 6.1. 모든 트레픽은 Gateway 보내라는 0.0.0.0/0 라우팅 테이블 생성

 6.2. 생성한 Routing Table을 Gateway와 연결

resource "aws_route_table" "eks_rota" {
  vpc_id = aws_vpc.eks_vpc.id

  route {
    cidr_block = var.rocidr
    gateway_id = aws_internet_gateway.eks_ig.id
  }

  tags = {
    Name = "${var.name}-rota"
  }
}

 

7. Routing Table Association

 7.1. 생성한 Routing Table과 Subnet을 연결

resource "aws_route_table_association" "eks_rotaass1" {
  count          = 2
  subnet_id      = aws_subnet.eksnet_ma[count.index].id
  route_table_id = aws_route_table.eks_rota.id
}

resource "aws_route_table_association" "eks_rotaass2" {
  count          = 2
  subnet_id      = aws_subnet.eksnet_work[count.index].id
  route_table_id = aws_route_table.eks_rota.id
}

 

8. Security 

 8.1. WorkNode와 Master Node가 통신할 TCP 443 Port 개방

resource "aws_security_group" "eks_secu" {
  name   = "${var.name}-cluster"
  vpc_id = aws_vpc.eks_vpc.id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = [var.rocidr]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [var.rocidr]
  }
  tags = {
    Name = "${var.name}-security"
  }
}

 

9. EKS Cluster

 9.1. EKS Cluster 생성하기

 9.2. 역할 부분 주의 필요

resource "aws_eks_cluster" "eks_clu" {
  name     = "eks-clu"
  role_arn = aws_iam_role.eks_clurole.arn

  vpc_config {
    subnet_ids              = concat(aws_subnet.eksnet_ma[*].id,aws_subnet.eksnet_work[*].id)
    endpoint_private_access = true
    endpoint_public_access  = true
    security_group_ids      = [aws_security_group.eks_secu.id]
  }
  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSClusterPolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKSVPCResourceController,
  ]
}

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["eks.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "eks_clurole" {
  name               = "eks-clurole"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSClusterPolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_clurole.name
}

# Optionally, enable Security Groups for Pods
# Reference: https://docs.aws.amazon.com/eks/latest/userguide/security-groups-for-pods.html
resource "aws_iam_role_policy_attachment" "eks-AmazonEKSVPCResourceController" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSVPCResourceController"
  role       = aws_iam_role.eks_clurole.name
}
/*
resource "aws_eks_addon" "eks_coredns" {
  cluster_name                = aws_eks_cluster.eks_clu.name
  addon_name                  = "coredns"
  addon_version               = "v1.10.1-eksbuild.2"
  resolve_conflicts_on_create = "OVERWRITE"
}
*/
resource "aws_eks_addon" "eks_cni" {
  cluster_name                = aws_eks_cluster.eks_clu.name
  addon_name                  = "vpc-cni"
  addon_version               = "v1.13.2-eksbuild.1"
  resolve_conflicts_on_create = "OVERWRITE"
}

resource "aws_eks_addon" "eks_proxy" {
  cluster_name                = aws_eks_cluster.eks_clu.name
  addon_name                  = "kube-proxy"
  addon_version               = "v1.27.3-eksbuild.1"
  resolve_conflicts_on_create = "OVERWRITE"
}

 

10. EKS WorkNode

 10.1. 1개의 WorkNode 생성

 10.2. 역할 부분 주의 

resource "aws_eks_node_group" "eks_node" {
  cluster_name    = aws_eks_cluster.eks_clu.name
  node_group_name = "${var.name}-node"
  node_role_arn   = aws_iam_role.eks_noderole.arn
  subnet_ids      = concat(aws_subnet.eksnet_ma[*].id, aws_subnet.eksnet_work[*].id)
  capacity_type   = "ON_DEMAND"
  disk_size       = 10
  instance_types  = ["t3.micro"]

  scaling_config {
    desired_size = 1
    max_size     = 2
    min_size     = 1
  }

  update_config {
    max_unavailable = 1
  }

  # Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
  # Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
  depends_on = [
    aws_iam_role_policy_attachment.eks-AmazonEKSWorkerNodePolicy,
    aws_iam_role_policy_attachment.eks-AmazonEKS_CNI_Policy,
    aws_iam_role_policy_attachment.eks-AmazonEC2ContainerRegistryReadOnly,
  ]
}

resource "aws_iam_role" "eks_noderole" {
  name = "eks-noderole"

  assume_role_policy = jsonencode({
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "ec2.amazonaws.com"
      }
    }]
    Version = "2012-10-17"
  })
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKSWorkerNodePolicy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_noderole.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEKS_CNI_Policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_noderole.name
}

resource "aws_iam_role_policy_attachment" "eks-AmazonEC2ContainerRegistryReadOnly" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.eks_noderole.name
}

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group

 

* 부록

변수 정의 파일

variable "region" {
  type    = string
  default = "ap-northeast-2"
}

variable "cidr" {
  type    = string
  default = "10.0.0.0/16"
}

variable "name" {
  type    = string
  default = "eks"
}

variable "rocidr" {
  type    = string
  default = "0.0.0.0/0"
}

 

자! 지금까지 수작업으로 고생하던 EKS를 자동화해 보았습니다.

나머지 작업들(namespace, deployment, service)도 완료해 보시기 바랍니다.

 

참고로 Addon중 Coredns는 설치하지 마시기 바랍니다.

성능저하문제로 EKS 생성을 방해합니다.

 

Have a nice day!