AWS: creating an EKS cluster

Top Tips

Stuff, perhaps not immediately relevant, but you’ll keep coming back to:

List contexts: kubectx

Switch contexts: `kubectx <your context>`

Namespaces:  `kubectl get pods -o yaml -n kube-system`

(e.g. if you run kubectl get pods and see nothing it may be ‘cos you’re using the wrong namespace – i.e. there are no pods in that namespace)

 

 

Notes and Guides:

Notes: EKS is only available in:

  • US West (Oregon) (us-west-2)
  • US East (N. Virginia) (us-east-1)
  • EU (Ireland) (eu-west-1)

Terraform guide:  https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html

(The Terraform code provided is here: https://github.com/terraform-providers/terraform-provider-aws/tree/master/examples/eks-getting-started )

and the AWS EKS guide: https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

 

Terraform notes:

  • TF code creates 2 m4.large instances based on the latest EKS Amazon Linux 2 AMI: Operator managed Kubernetes worker nodes for running Kubernetes service deployments
  • Full code: https://github.com/terraform-providers/terraform-provider-aws/tree/master/examples/eks-getting-started

 

AWS EKS notes

You’ll need:

  • aws-iam-authenticator

Don’t use the instructions given on https://github.com/kubernetes-sigs/aws-iam-authenticator unless you want to waste half an hour of your time figuring out why it doesn’t work. I got this error: https://stackoverflow.com/questions/53344191/running-go-gives-me-go-clang-error-no-input-files

Use the instructions here: https://docs.aws.amazon.com/eks/latest/userguide/configure-kubectl.html

i.e. curl -o aws-iam-authenticator https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-07-26/bin/darwin/amd64/aws-iam-authenticator

  • helm
  • kubectl

 

Name of cluster: in AWS console or use:

aws eks list-clusters

 

To use kubectl:

aws eks update-kubeconfig --name <name of cluster>

This will add the config to your ~/.kube/config.

Checking:

1. You can check this is in your config with:

  • kubectl config view

See also Kubernetes: kubectl

 

Note:  aws cli version <= 1.15.53 does not have this. Upgrade AWS CLI, with:`pip install awscli –upgrade –user`

https://docs.aws.amazon.com/cli/latest/userguide/installing.html

Typical problems when upgrading AWS CLI:

aws --version
aws-cli/1.11.10 Python/2.7.10 Darwin/17.7.0 botocore/1.4.67

pip install awscli --upgrade --user
Collecting awscli
  Downloading https://files.pythonhosted.org/packages/a6/da/c99b10bfc509cbbea520886d2e8fe0e738e3ce22e2f528381f3bb2229433/awscli-1.16.57-py2.py3-none-any.whl (1.4MB)
...
Successfully installed awscli-1.16.57 botocore-1.12.47

aws --version
aws-cli/1.11.10 Python/2.7.10 Darwin/17.7.0 botocore/1.4.67

You’ve probably got a PATH problem.

Check you haven’t got an older version at /usr/local/bin

 

2. And that you can see pods in your cluster with:

kubectl get all -n kube-system

E.g. I got this back:

NAME                          READY   STATUS    RESTARTS   AGE
pod/kube-dns-fcd468cb-8fhg2   0/3     Pending   0          41m

NAME               TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)         AGE
service/kube-dns   ClusterIP   172.20.0.10   <none>        53/UDP,53/TCP   41m

NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/aws-node     0         0         0       0            0           <none>          41m
daemonset.apps/kube-proxy   0         0         0       0            0           <none>          41m

NAME                       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kube-dns   1         1         1            0           41m

NAME                                DESIRED   CURRENT   READY   AGE
replicaset.apps/kube-dns-fcd468cb   1         1         0       41m

 

Debugging:

Some more information on debugging Pods

kubectl get events --all-namespaces

shows

kube-system 1m 1h 245 kube-dns-fcd468cb-8fhg2.156899dbda62d287 Pod Warning FailedScheduling default-scheduler no nodes available to schedule pods

and

kubectl get nodes
No resources found.

so ssh into one of the nodes and run journalctl

You’ll need to add your ssh key to the node and get the public IP address. Then:

ssh -i ~/path/to/key ec2-user@public.ip.address

 

StackOverflow post: https://stackoverflow.com/questions/53381739/kube-system-pod-warning-failedscheduling-default-scheduler-no-nodes-available-t

 

The trick to solving this is the output that’s generated by Terraform needs to be applied.

i.e. copy `config_map_aws_auth` which, for me, looked like:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-auth
  namespace: kube-system
data:
  mapRoles: |
    - rolearn: arn:aws:iam::<owner id>:role/terraform-eks-demo-node
      username: system:node:{{EC2PrivateDNSName}}
      groups:
        - system:bootstrappers
        - system:nodes

into a file,  config_map_aws_auth.tf.output and apply as is:

kubectl apply -f config_map_aws_auth.tf.output

The {{EC2PrivateDNSName}} is parsed by one of the Kubernetes controllers.

More on this issue in #office-hours – https://kubernetes.slack.com/archives/C6RFQ3T5H/p1542812144088800

 

Issues

Warning FailedScheduling – default-scheduler no nodes available to schedule pods

error creating EKS Cluster: InvalidParameterException: Error in role params

AWS EKS: An error occurred (ResourceNotFoundException) when calling the DescribeCluster operation: No cluster found for name

 

 

Screencast

https://asciinema.org/a/zYFCtGrXSqJaLHybKwq6V9rFF

 

Leave a Reply

Your email address will not be published. Required fields are marked *