Billing: reducing costs on AWS, Kubernetes, etc

Our monthly AWS bill is millions of dollars per year. So, we’re always trying to reduce costs.

Here are a few tips – some of these are Kubernetes but mainly AWS:

  • reduce size of pods
  • get the right node size and type
  • get the right instance class
  • don’t over-provision storage
  • clean up lost resources

https://medium.com/@dyachuk/why-do-kubernetes-clusters-in-aws-cost-more-than-they-should-fa510c1964c6

 

Cost Optimization is also one of the five pillars of the Well-Architected Framework from AWS: https://aws.amazon.com/architecture/well-architected/

i.e. https://d1.awsstatic.com/whitepapers/architecture/AWS-Cost-Optimization-Pillar.pdf

 

Billing Dashboard AWS: https://console.aws.amazon.com/billing/home

 

Wizards: why they suck

You see it everywhere – from Windows dialogs to AWS.

Wizards are supposed to simplify a task by focusing on just a few specific items.

But the problem is that you can end up losing track of the big picture whilst you’re traversing a long wizard.

West Midlands Space Centre

Aim: launch at least 1 satellite from the West Midlands

How: using Rockoon technology

Phases:

  1. launch high altitude balloon
  2. launch rocket
  3. launch rockoon
  4. launch rockoon with satellite
  5. launch rockoon with satellite to LEO
  6. launch rockoon with satellite to the Moon

Phase 1

Pico tracker: http://www.loratracker.uk/?p=30

Useful links:

UKHAS:

BIS West Midlands:

Black Knights Rocket club:

 

 

 

Kubernetes: EKS – Warning FailedScheduling – default-scheduler no nodes available to schedule pods

I spun up an EKS cluster and, when I ran:

kubectl get pods -n kube-system

got

 

More detail needed:

kubectl describe pods -n kube-system

Warning FailedScheduling 2m13s (x345 over 102m) default-scheduler no nodes available to schedule pods

 

Looking in EC2 Instances, I could see 2 EKS nodes running with the correct AMI IDs.

But Kubernetes can’t see these nodes. E.g.

kubectl get nodes

No resources found.

Note: low level objects like nodes aren’t namespaced – https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#not-all-objects-are-in-a-namespace

 

 

 

 

 

Kubernetes: Imperative vs Declarative

TLDR: use Declarative. E.g. Helm charts

 

Imperative commands

  • objects are created and managed/modified using the CLI
  • all operations are done on live objects

E.g.

kubectl create ns ghost
kubectl create quota blog –hard=pods=1 -n ghost
kubectl run ghost –image=ghost -n ghost
kubectl expose deployments ghost –port 2368 –type LoadBalancer -n ghost

Or

kubectl create service clusterip foobar --tcp=80:80

To modify any of the objects you can use the kubectl edit command or use any of the convenience wrappers. For example to scale the deployment do:

kubectl scale deployment ghost --replicas 2 -n ghost

https://kubernetes.io/docs/concepts/overview/object-management-kubectl/imperative-command/

 

Declarative mode

Use a YAML file and run something like:

kubectl apply -f <object>.yaml

https://kubernetes.io/docs/concepts/overview/object-management-kubectl/declarative-config/

 

More: https://medium.com/bitnami-perspectives/imperative-declarative-and-a-few-kubectl-tricks-9d6deabdde

Fedex sucks – badly

I’ve got a delivery from the US.

I wasn’t at home when it arrived. There wasn’t a card but according to the tracking number, it was supposed to be delivered the next day by 6pm.

I stayed in all day. I was pretty sure it wouldn’t arrive.

By 4pm I was sure and called and asked. They said it was on the van.

By 5pm their phone lines stopped answering.

By 6:40pm no parcel and the website said their office hours were 9am to 6pm.

I rang the US International office. They said they’d have to put me through to the UK office. I explained that this would be shut. But they gave me the UK International number anyway.

Which turned out to be wrong. It was an 0845 number and I had to redial with 0345. Finally I got through to reach an answering maching which said they were shut.

And they disconnected.

Why was I so sure it wouldn’t arrive?

‘cos this happened exactly the same last time.