Billing: reducing costs on AWS, Kubernetes, etc

Our monthly AWS bill is millions of dollars per year. So, we’re always trying to reduce costs.

Here are a few tips – some of these are Kubernetes but mainly AWS:

  • reduce size of pods
  • get the right node size and type
  • get the right instance class
  • don’t over-provision storage
  • clean up lost resources

https://medium.com/@dyachuk/why-do-kubernetes-clusters-in-aws-cost-more-than-they-should-fa510c1964c6

 

Cost Optimization is also one of the five pillars of the Well-Architected Framework from AWS: https://aws.amazon.com/architecture/well-architected/

i.e. https://d1.awsstatic.com/whitepapers/architecture/AWS-Cost-Optimization-Pillar.pdf

 

Billing Dashboard AWS: https://console.aws.amazon.com/billing/home

 

Kubernetes: Dummy Pod

So we can test kubectl let’s create a dummy pod.

Apply with:

kubectl apply -f pod.yaml

and test with:

kubectl get pods

 

I get:

 

The logs won’t show anything ‘cos it hasn’t started yet.

Let’s describe the pod.

k describe pod w-dummy-pod

Memory pressure.

Let’s give minikube more memory.

minikube config set memory 2048

then

and we can exec in with:

k exec -it w-dummy-pod bash

 

Let’s check we can cp a file in:

echo "test" > test.txt

Prove it doesn’t exist in the Pod by:

k exec -it w-dummy-pod bash

and ls

Then:

k cp test.txt w-dummy-pod:/testing.txt

and then exec in and ls.

 

 

 

 

 

 

DockerFile: WordPress

Let’s take a look at Docker‘izing WordPress.

 

The Docker pull command is:

docker pull wordpress

pull: pulls an image or a repository from a registry. It doesn’t run it. It just means you have the image locally.

https://docs.docker.com/engine/reference/commandline/pull/

 

You can actually run the image using:

docker run --name some-wordpress --link some-mysql:mysql -d wordpress

some-wordpress is going to be the name of the container.

--link is a bit old school. It connects one container to another. i.e. MySQL to WordPress. Nowadays we use user-defined networks – e.g. overlays. https://docs.docker.com/network/links/

WordPress Docker Repo: https://hub.docker.com/_/wordpress/

 

However, before you can run it you’ll need MySQL. So:

docker pull mysql:5.7.24

(Aside: why not use mysql or mysql:latest? ‘cos MySQL 8 changed the password authentication method. See below.)

and then run it with:

docker run --name test-mysql -e MYSQL_ROOT_PASSWORD=test -d mysql:latest

--name is the name you’re giving to the container,

MYSQL_ROOT_PASSWORDis an environment variable that you set which is read in the container. Note: with MySQL this is done programmatically via docker-entrypoint.sh – https://github.com/docker-library/mysql/blob/696fc899126ae00771b5d87bdadae836e704ae7d/8.0/docker-entrypoint.sh . For more on ARG, Environment variables: https://stackoverflow.com/questions/53592879/dockerfile-and-environment-variable/53593826#53593826

and -d means detach.

To specify a tagged version just add it after the image using a colon. E.g. mysql:5.7.24.

 

Once you’ve run it you can exec in with

docker exec -it test-mysql bash

or check logs with:

docker logs test-mysql

E.g.

2018-12-03T14:24:57.091306Z 0 [Note] mysqld: ready for connections.
Version: '5.7.24' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server (GPL)

 

Or even mysql in via another MySQL container using:

docker run -it --link test-mysql:mysql --rm mysql sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"'

https://hub.docker.com/_/mysql/

 

So, getting back to WordPress let’s run it now with:

docker run --name test-wordpress --link test-mysql:mysql -d wordpress

and check the logs with:

You should then be able to access your WordPress site on http://localhost:8080

 

Errors

Conflict. The container name “/test-wordpress” is already in use by container

You run

docker run --name test-wordpress --link test-mysql:mysql -d wordpress

and get:

docker: Error response from daemon: Conflict. The container name "/test-wordpress" is already in use by container "0ea70abdaf306d896eb71f3ab585961359f27af23a243b81370bf407d3dd846d". You have to remove (or rename) that container to be able to reuse that name.

You’ve already got a container with that name.

Remove it with: docker rm test-wordpress

https://stackoverflow.com/questions/31676155/docker-error-response-from-daemon-conflict-already-in-use-by-container

This might happen if the container exited and you try and relaunch it.

 

Site can’t be reached

You plug http://localhost:8080 into the web browser but get:

Checking the WordPress logs with docker logs test-wordpress I can see:

however this is a secondary problem. Why are we getting this?

‘cos MySQL 8 introduced a different type of authentication – https://github.com/docker-library/wordpress/issues/313

If you are getting this then you need to use: docker pull mysql:5.7.24 or use a different auth method.

 

Back to the problem at hand – we should still be able to see a (non-functioning) WordPress site on that port. i.e. Apache should be running.

Let’s just do a sanity check:

This 0.0.0.0:8080->80/tcp means the docker host port 8080 is mapped to the container port 80.

https://stackoverflow.com/questions/41798284/understanding-docker-port-mappings

so http://localhost:8080 is correct.

 

It seems the problem really was the lack of MySQL. Using 5.7.24 and looking at the WordPress logs showed (for a successful installation):

 

Installing MySQL and WordPress in under a minute:

https://asciinema.org/a/nW3l0A1hi6wzMde4RUQ1QVPsc?speed=3