Tuesday, July 7, 2020

K8S-AWS-EKS-ECR-SPRING_BOOT-DEPLOYMENT-WITH-TERRAFORM-AND-BASH-SHELL

Howdy?

This blog post shows how to run eks on aws with terraform and bash shell. And also simple java web application deployment with kubectl.


  1.  Clone my git  repo that includes tf and shell files for provisioning eks. And terraform apply with shell
  2.  JAVA SPRING-BOOT DEPLOYMENT.
  3.  CONCLUSION



Prerequisites)
*Basic linux(unix) commands.
*aws cli(with privilege eks and ecr,etc)
oyj@laptopa:~$ aws --version
aws-cli/2.0.23 Python/3.7.3 Linux/5.4.0-40-generic botocore/2.0.0dev27
oyj@laptopa:~$
-privilege test. Just no error is ok

   oyj@laptopa:~/prac/aws/eks/test$ aws eks list-clusters

*Ubuntu box
oyj@laptopa:$ uname -a
Linux laptopa 5.4.0-40-generic #44-Ubuntu SMP Tue Jun 23 00:01:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
oyj@laptopa:~/$ cat /etc/issue

*git
oyj@laptopa:~$ which git
/usr/bin/git
oyj@laptopa:~$

*Terraform install. If you ansible,then I hava playbook file. Help yourself.(https://raw.githubusercontent.com/ohyoungjooung2/my_ansible/master/tera_install.yaml)
oyj@laptopa:~$ terraform --version
Terraform v0.12.28

1. Clone my git  repo that includes tf and shell files for provisioning eks.
And terraform apply with shell.

oyj@laptopa:~/prac/aws/eks/test$ git clone git@github.com:ohyoungjooung2/terraform-awseks.git
Cloning into 'terraform-awseks'...
remote: Enumerating objects: 88, done.
remote: Counting objects: 100% (88/88), done.
remote: Compressing objects: 100% (61/61), done.
remote: Total 88 (delta 33), reused 67 (delta 20), pack-reused 0
Receiving objects: 100% (88/88), 98.86 KiB | 568.00 KiB/s, done.
Resolving deltas: 100% (33/33), done.

oyj@laptopa:~/prac/aws/eks/test$ cd terraform-awseks/
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ terraform init

Initializing the backend...

Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.69.0...

Terraform has been successfully initialized!

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ cat apply.sh
#!/usr/bin/env bash
terraform apply -auto-approve
sleep 1
terraform apply -auto-approve

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ bash apply.sh
aws_iam_role.node-role: Creating...
aws_key_pair.eks-nat: Creating...
aws_iam_role.eksrole: Creating...
aws_vpc.eks-vpc: Creating...
aws_key_pair.eks-nat: Creation complete after 0s [id=eks-nat]
aws_vpc.eks-vpc: Creation complete after 2s [id=vpc-0746a9ef2902c81b2]
aws_internet_gateway.eks-gw: Creating...
--...............
BLABLABLABLABLABLA

   name: kubernetes
contexts:
-  context:
      cluster: kubernetes
      user: aws
   name: aws
   current-context: aws
   kind: Config
   preferences: {}
   users:
   - name: aws
     user:
       exec:
         apiVersion: client.authentication.k8s.io/v1alpha1
         command: aws-iam-authenticator
         args:
           - "token"
           - "-i"
           - "terraform-eks-demo"
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ terraform state list
aws_eks_cluster.eks-cluster
aws_eks_node_group.terraform-demo-cluster-grp
aws_iam_instance_profile.node-profile
aws_iam_role.eksrole
aws_iam_role.node-role
aws_iam_role_policy_attachment.eksrole-AmazonEKSClusterPolicy
aws_iam_role_policy_attachment.eksrole-AmazonEKSServicePolicy
aws_iam_role_policy_attachment.node-AmazonEC2ContainerRegistryReadOnly
aws_iam_role_policy_attachment.node-AmazonEKSWorkerNodePolicy
aws_iam_role_policy_attachment.node-AmazonEKS_CNI_Policy
aws_instance.nat
aws_internet_gateway.eks-gw
aws_key_pair.eks-nat
aws_route_table.eks-rt
aws_route_table.eks-rt-nat
aws_route_table_association.eks-rt-association1
aws_route_table_association.eks-rt-association2
aws_route_table_association.eks-rt-natinst-association1
aws_route_table_association.eks-rt-natinst-association2
aws_security_group.eks-cluster-sg
aws_security_group.nat
aws_security_group.node-sg
aws_security_group.private
aws_security_group_rule.cluster-ingress-node-https
aws_security_group_rule.node-ingress-cluster
aws_security_group_rule.node-ingress-self
aws_subnet.eks-node-subnet-pri1
aws_subnet.eks-node-subnet-pri2
aws_subnet.eks-node-subnet-pub1
aws_subnet.eks-node-subnet-pub2
aws_vpc.eks-vpc

*kubectl install
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ bash kubectl_config.sh
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41.0M  100 41.0M    0     0  4264k      0  0:00:09  0:00:09 --:--:-- 4979k
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ which kubectl
/usr/local/bin/kubectl

*Kube config to authenticate.
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ bash eks-kctl.sh
Updated context arn:aws:eks:ap-northeast-2:494307375889:cluster/terraform-eks-demo in /home/oyj/.kube/config


*If you see something like below, CONGRATULATIONS!. CLUSTER IS UP.

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ kubectl get nodes
NAME                                           STATUS   ROLES    AGE   VERSION
ip-10-0-0-23.ap-northeast-2.compute.internal   Ready    <none>   13m   v1.16.8-eks-fd1ea7
ip-10-0-1-59.ap-northeast-2.compute.internal   Ready    <none>   13m   v1.16.8-eks-fd1ea7

*RUNNING PO IN KUBE-SYSTEM NAMESPACES.
yj@laptopa:~/prac/aws/eks/test/terraform-awseks$ kubectl get po -n kube-system
NAME                       READY   STATUS    RESTARTS   AGE
aws-node-84tpx             1/1     Running   0          44m
aws-node-thfpb             1/1     Running   0          44m
coredns-7b64b848fd-5xdgf   1/1     Running   0          55m
coredns-7b64b848fd-zd7j4   1/1     Running   0          55m
kube-proxy-rfq76           1/1     Running   0          44m
kube-proxy-zrl9l           1/1     Running   0          44m


*CLUSTER LIST
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ aws eks list-clusters
CLUSTERS        terraform-eks-demo

*NODEGROUPS
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ aws eks list-nodegroups --cluster-name terraform-eks-demo
NODEGROUPS      terraform-demo-cluster-grp


*GENERATED WORKER NODE INSTANCES.
*CLUSTER DESCRIPTION.




2. JAVA SPRING-BOOT DEPLOYMENT.

  *We need to install maven. On ubuntu this is very easy.
  *Also need to install docker to create docker image.

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ sudo apt install maven
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ sudo apt install docker -y
oyj@laptopa:~$ docker --version
Docker version 19.03.8, build afacb8b7f0



oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ mvn --version
Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 11.0.7, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-40-generic", arch: "amd64", family: "unix"

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ git clone https://github.com/ohyoungjooung2/actuator-sample.git
Cloning into 'actuator-sample'...
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 217 (delta 6), reused 18 (delta 4), pack-reused 196
Receiving objects: 100% (217/217), 68.22 KiB | 335.00 KiB/s, done.
Resolving deltas: 100% (112/112), done.


*Mvn package
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ mvn package -B

--It will take a while if you do this first time,esp, in slow internet speed.
  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.6.RELEASE)

[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.960 s
[INFO] Finished at: 2020-07-07T02:43:58+09:00
[INFO] ------------------------------------------------------------------------

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ ./mvnw spring-boot:run
[INFO] Scanning for projects...
[INFO]
[INFO] --------------------< com.example:actuator-sample >---------------------
[INFO] Building actuator-sample 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] >>> spring-boot-maven-plugin:2.1.6.RELEASE:run (default-cli) > test-compile @ actuator-sample >>>
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ actuator-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ actuator-sample ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ actuator-sample ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/oyj/prac/aws/eks/test/terraform-awseks/actuator-sample/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ actuator-sample ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] <<< spring-boot-maven-plugin:2.1.6.RELEASE:run (default-cli) < test-compile @ actuator-sample <<<
[INFO]
[INFO]
[INFO] --- spring-boot-maven-plugin:2.1.6.RELEASE:run (default-cli) @ actuator-sample ---

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.6.RELEASE)


2020-07-07 20:30:46.159  INFO 15416 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 2 endpoint(s) beneath base path '/actuator'
2020-07-07 20:30:46.327  INFO 15416 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8082 (http) with context path ''
2020-07-07 20:30:46.340  INFO 15416 --- [           main] c.e.a.ActuatorSampleApplication          : Started ActuatorSampleApplication in 5.899 seconds (JVM running for 15.83)


* Docker build this web app.
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ docker build -t java-actuator .
Sending build context to Docker daemon  18.58MB
Step 1/4 : FROM openjdk:8u111-jdk-alpine
8u111-jdk-alpine: Pulling from library/openjdk

Removing intermediate container ecda127ace58
 ---> c5c58fdee62f
Successfully built c5c58fdee62f
Successfully tagged java-actuator:latest

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$  docker tag java-actuator:latest 494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator:latest


*CREATE ECR REPO AND PUSH IMAGE.
 oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ aws ecr create-repository --repository-name java-actuator2 --image-tag-mutability IMMUTABLE
REPOSITORY      2020-07-07T20:41:39+09:00       IMMUTABLE       494307375889    arn:aws:ecr:ap-northeast-2:494307375889:repository/java-actuator2       java-actuator2  494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2
IMAGESCANNINGCONFIGURATION      False

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ docker tag java-actuator:latest 494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2:latest

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ aws ecr get-login-password --region ap-northeast-2 | docker login --username AWS --password-stdin 494307375889.dkr.ecr.ap-northeast-2.amazonaws.com
WARNING! Your password will be stored unencrypted in /home/oyj/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded


oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ docker push 494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2:latest
The push refers to repository [494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2]
1aae1ae5d0fa: Pushed
a1e7033f082e: Pushed
78075328e0da: Pushed
9f8566ee5135: Pushed
latest: digest: sha256:25cca98c896a16308d8c22ad1363083d4c7089126e36ef7cd6e4449aafb09216 size: 1159


oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ aws ecr describe-repositories | grep actuator2
REPOSITORIES    2020-07-07T20:41:39+09:00    IMMUTABLE    494307375889    arn:aws:ecr:ap-northeast-2:494307375889:repository/java-actuator2    java-actuator2    494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ aws ecr list-images --repository-name java-actuator2
IMAGEIDS        sha256:25cca98c896a16308d8c22ad1363083d4c7089126e36ef7cd6e4449aafb09216 latest


*FINALLY LET'S DEPLOY WITH KUBECTL.

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ cat actuator-svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: actuator-sample
  labels:
    app: actuator-sample
spec:
  ports:
    - port: 8082
  selector:
    app: actuator-sample
    #type: NodePort
  type: LoadBalancer
  #ports:
  #- port: 8073
  #  nodePort: 32338
  #  targetPort: 8082
  #  protocol: TCP
  #  name: http
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ kubectl create -f actuator-svc.yaml
service/actuator-sample created
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ kubectl get svc
NAME              TYPE           CLUSTER-IP      EXTERNAL-IP                                                                   PORT(S)          AGE
actuator-sample   LoadBalancer   172.20.56.110   a0331390890114e80a1d0c8dbd3157d3-401330172.ap-northeast-2.elb.amazonaws.com   8082:32619/TCP   14s
kubernetes        ClusterIP      172.20.0.1      <none>                                                                        443/TCP          80m

--This will create load balancer(classic-default)--


#K8S DEPLOYMENT THE APP actuator-sample very simple java web.
  oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ cat actuator-dp-ecr.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: actuator-sample
  namespace: default
  labels:
     app: actuator-sample
spec:
  replicas: 2
  selector:
    matchLabels:
      app: actuator-sample
  template:
     metadata:
       labels:
         app: actuator-sample
     spec:
       containers:
       - name: actuator-sample
         image: 494307375889.dkr.ecr.ap-northeast-2.amazonaws.com/java-actuator2:latest
         imagePullPolicy: Always
         ports:
         - containerPort: 8082
       #imagePullSecrets:
       #  - name: regcred

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ kubectl create -f actuator-dp-ecr.yaml
deployment.apps/actuator-sample created
#BELOW SERVICE IS "IN SERVICE" STATE. GOOD!
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ kubectl get po
NAME                               READY   STATUS    RESTARTS   AGE
actuator-sample-66498f79bf-nnbnz   1/1     Running   0          6m9s
actuator-sample-66498f79bf-x8wgq   1/1     Running   0          6m9s




#FINAL CHECK WITH WEB BROWSER.
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ telnet a62d7280fcdfb43f78fe28f8c6e1794b-1487394716.ap-northeast-2.elb.amazonaws.com 8082
Trying 52.79.215.222...
Connected to a62d7280fcdfb43f78fe28f8c6e1794b-1487394716.ap-northeast-2.elb.amazonaws.com.
Escape character is '^]'.
^CConnection closed by foreign host.
#LET'S REMOVE THIS DEVELOPMENT STAGE FOR FRUGAL?
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ kubectl delete -f actuator-dp-ecr.yaml -f actuator-svc.yaml
deployment.apps "actuator-sample" deleted
service "actuator-sample" deleted

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks/actuator-sample$ cd ..
oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ cat delete.sh
#!/usr/bin/env bash
terraform destroy --force
sleep 1
terraform destroy --force

oyj@laptopa:~/prac/aws/eks/test/terraform-awseks$ bash delete.sh
#THIS WILL TAKE SOME TIME.
BLABLA...


Destroy complete! Resources: 10 destroyed.




CONCLUSION)
With terraform and bash(sometimes ansible like tool) surely help us
to simplify(?) the architecture of AWS EKS. And aws eks is more simple than the one that deploys on bare metals.

Another good point is that with this development, final production deployment shall be consise and easier than no pre configured "INFRA CODE"

THANKS FOR READING.


No comments:

Post a Comment