Fully automated canary deployments in Kubernetes

See here for a Hello World example using Codefresh.

and a manual Canary deployment: https://github.com/codefresh-io/k8s-canary-deployment

which uses a bash script (k8s-canary-rollout.sh) with parameters.


Otherwise proceed to:


This webinar also shows with / without Istio and using Helm for deployments: https://codefresh.io/webinars/istio-canary-deployment-with-helm-and-codefresh/




k8s deployment strategies

1. start local Kubernetes v1.10.0 cluster…


If it hangs on:

Starting cluster components...

You can see what’s going on with:

minikube logs

which spits out thousands of lines of logs.

Annoyingly minikube logs -f does not work even though it’s implemented internally: https://github.com/kubernetes/dashboard/issues/1083

See Installing Kubernetes: Minikube for a solution.


2. helm init

3. Install Prometheus

which outputs

Note: it mentions ContainerCreating – you can check on the current status with:

helm list

This shows you the state of the pod you’ve just created:

prometheus 1 Thu Nov 1 13:22:45 2018 DEPLOYED prometheus-7.0.0 2.3.2 monitoring

which had those 4 containers:

prometheus-node-exporter-mfgcj 0/1 ContainerCreating 0 1s prometheus-alertmanager-99f6bfbcc-b8hkc 0/2 ContainerCreating 0 0s prometheus-kube-state-metrics-6584885ccf-fkkxc 0/1 ContainerCreating 0 0s prometheus-pushgateway-d5fdc4f5b-m7kzj 0/1 ContainerCreating 0 0s prometheus-server-86887bb56b-sjwhq 0/2 Pending 0 0s




Source: https://github.com/ContainerSolutions/k8s-deployment-strategies

Helm Charts

Charts describe a set of Kubernetes resources – e.g. a full web app stack with HTTP servers, databases, caches, etc.

requirements.yamldefines dependencies using:

  • name
  • version
  • repository

Tags: like Ansible

Condition: enabled / disabled – always override tags.

See https://github.com/helm/helm/blob/master/docs/charts.md


Manage charts with helm:

  • create – creates chart
  • package – packages
  • lint – checks formatting


Getting started with Helm:

1. check kubectl config – i.e. using local minikube

kubectl config view | grep current

2. start helm

helm init



Installing MySQL as a Helm Chart

Running helm install stable/mysql

(which uses: https://github.com/helm/charts/tree/master/stable/mysql )

Let’s test we can connect to MySQL.

From the output, let’s get the MySQL password:

kubectl get secret --namespace default queenly-seahorse-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo


Note: you could have got the pod name with:

kubectl get pods

Now exec into MySQL with:

kubectl exec -it queenly-seahorse-mysql-6dc964999c-h4w54 bash

Install MySQL client:

apt-get update && apt-get install mysql-client -y --force-yes

and connect with:

mysql -h localhost -p



More on:

  • kubectl commands here: Kubernetes: kubectl
  • MySQL Notes here: https://github.com/helm/charts/blob/master/stable/mysql/templates/NOTES.txt

Installing WordPress as a Helm Chart

List with

helm list

and delete with

helm delete my-release




Error: no available release name found





Error: Get!D(MISSING)TILLER: dial tcp i/o timeout

When you do a helm list

From https://github.com/helm/helm/issues/3055#issuecomment-385371327


kubectl delete the tiller service and deployment.)

So: kubectl delete tiller-deploy-6fd8d857bc-fp5s2
error: resource(s) were provided, but no name, label selector, or –all flag specified

kubectl list
Error: unknown command “list” for “kubectl”

This suggests deleting tiller using

helm reset

but this gives:



and helm ls

Another, not very helpful, issue on why you can’t delete tiller:


Checking tiller:

kubectl get deploy -n kube-system


To see pods in kube-system

kubectl get pods –namespace kube-system


tiller-deploy-6fd8d857bc-fp5s2 1/1 Running 7 8d



Tiller namespaces and RBAC

Namespaces are for different environments. E.g. production, staging.


RBAC and Service Accounts: 



Further reading

Use ksonnet to generate Kubernetes configurations from Helm Charts



Kubernetes: helm

Getting it up and running

Install with brew install kubernetes-helm


Main concepts:

1. Chart: Helm package – contains all the resource definitions to run an application in a Kubernetes cluster.

See Helm Charts for examples of using Helm

2. Repository: where charts are stored

3. Release: an instance of a chart in a Kubernetes cluster. E.g. with a MySQL chart, you can have 2 databases running in a cluster by installing the chart twice. Each is its own release with its own release name.


Helm has two parts: the client (helm) and the server (tiller).

Tiller runs inside the Kubernetes cluster



helm – outputs commands available

helm version – outputs client / server version.

helm init – runs helm.

helm init --upgrade– to upgrade Tiller.



Error: Get dial tcp connect: connection refused

it’s probably ‘cos you need to start Kubernetes. E.g. with minikube use:

minikube start


Error: could not find a ready tiller pod

if you run kubectl -n kube-system get po, check you see a tiller-deploy pod available there. Then, given the pod `tiller-deploy-6fd8d857bc-fp5s2

kubectl logs tiller-deploy-6fd8d857bc-fp5s2

Error from server (NotFound): pods "tiller-deploy-6fd8d857bc-fp5s2" not found

Solution to this is to use


kubectl logs --namespace kube-system tiller-deploy-6fd8d857bc-fp5s2

which says:




version is now working. But unsure if it was down to the earlier command I ran:

minikube addons enable registry-creds


Error: could not find tiller

Need to run helm init.




For more on Helm Charts see: Helm Charts