Migrating Helm Charts v2 to v3
Helm is one of the most popular package manager for Kubernetes. Basically, Helm takes the effort out of making sharable, packaged, template files that can easily be dropped into other Kubernetes Clusters without a lot of effort.
Helm defines a structure and a convention for a software package identified by different layers of ‘YAML templates’ and a separate layer called ‘values’ that lets you change the templates. This allows the separation of generic configuration with deployment-specific values. This whole package is called as ‘Helm Charts’
Back in November 2019, Helm Team announced the first stable release of Helm 3. The internal implementation of Helm 3 has changed considerably from Helm 2. The most apparent change is the removal of Tiller, but there are several other changes that can be checked here. This article provides a step by step guide of how to migrate from Helm v2 to v3 using Helm2to3 plugin.
One of the most important aspects of upgrading to a new major release of Helm is the migration of data. This is especially true of Helm v2 to v3 considering the architectural changes between the releases. The 2to3 plugin helps with this migration by supporting:
- Migration of Helm v2 configuration.
- Migration of Helm v2 releases.
- Clean up Helm v2 configuration, release data and Tiller deployment.
The article also assumes that you have familiarity and hands-on experience with Kubernetes (K8s) and already have a Kubernetes Cluster already setup. For those who are not familiar with Kubernetes (K8s), it is an open-source platform for managing containerized workloads and services. Please check out all the learning resources available on the K8s website and I also recommend you try out simple K8s tutorials.
I start with a basic 3-node K8s cluster setup on Google Cloud Platform (GCP) using Google Kubernetes Engine (GKE), and Helm v2.16.3 is installed.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ helm version
Client: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
$As you know, in Helm v2 architecture, a Helm client interacts with the ‘Tiller’ pod which acts as a server that runs inside your Kubernetes cluster. Tiller manages installations of your Helm Charts (i.e. K8s package). The following output shows a running tiller Pod and that there are no application Pods.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 13m
$
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ kubectl get deploy,svc tiller-deploy -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/tiller-deploy 1/1 1 1 81s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/tiller-deploy ClusterIP 10.51.240.136 <none> 44134/TCP 80s
$Also, there is no helm chart/application release currently installed.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ helm list
$Let’s start by deploying a basic NGINX application.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ helm install helm_example/
NAME: kneeling-zebra
LAST DEPLOYED: Thu Mar 12 07:16:08 2020
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Deployment
NAME AGE
nginx-deploy-example 0s
==> v1/Pod(related)
NAME AGE
nginx-deploy-example-78c668d7fb-ffg6x 0s
nginx-deploy-example-78c668d7fb-gr7g9 0s
nginx-deploy-example-78c668d7fb-t4pxw 0s
==> v1/Service
NAME AGE
nginx-svc-example 0s
NOTES:
1. Get the application URL by running these commands:
export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=helm_example,app.kubernetes.io/instance=kneeling-zebra" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl port-forward $POD_NAME 8080:80
$Based on my manifest files, K8s creates a deployment with 3 replicas of NGINX Pods and finally creates a K8s Service of type ClusterIP. My goal of deploying such a basic application is to play with Helm commands and not test the application itself.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deploy-example-78c668d7fb-ffg6x 1/1 Running 0 39s
pod/nginx-deploy-example-78c668d7fb-gr7g9 1/1 Running 0 39s
pod/nginx-deploy-example-78c668d7fb-t4pxw 1/1 Running 0 39s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 14m
service/nginx-svc-example ClusterIP 10.51.245.227 <none> 80/TCP 39s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deploy-example 3/3 3 3 39s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deploy-example-78c668d7fb 3 3 3 39s
$After the successful installation, the new Helm release can be seen under ‘helm list’ -
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ helm2 list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
kneeling-zebra 1 Thu Mar 12 07:16:08 2020 DEPLOYED helm_example-0.1.0 1.0 default
$Now that a basic Chart is installed, let us proceed with steps for Helm 3 installation followed by the Helm2to3 migration plugin. I copy my existing Helm v2 binary and call it ‘helm2’. Later on, this will be very useful for differentiating Helm 2 vs 3 binary.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ sudo cp /usr/local/bin/helm /usr/local/bin/helm2
$Next, install Helm v3. At the end of the installation, you can notice that the helm v3 is installed with a generic binary name ‘helm’. The previous step of copying the v2 binary and calling it ‘helmv2’ might make more sense now.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ curl -L https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6794 100 6794 0 0 56165 0 --:--:-- --:--:-- --:--:-- 56616
Helm v3.1.1 is available. Changing from version <no value>.
Downloading https://get.helm.sh/helm-v3.1.1-linux-amd64.tar.gz
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
$shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2 (smulkutk-project-1)$ helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
$
We shall use Helm2to3 plugin to migrate the basic NGINX application release from Helm v2 to v3.
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2tov3 (smulkutk-project-1)$ cat install_helm2to3_plugin.sh
helm plugin install https://github.com/helm/helm-2to3.git
$
shreyans_mulkutkar@cloudshell:~/my_utilities/helmv2tov3 (smulkutk-project-1)$ ./install_helm2to3_plugin.sh
Downloading and installing helm-2to3 v0.4.1 ...
https://github.com/helm/helm-2to3/releases/download/v0.4.1/helm-2to3_0.4.1_linux_amd64.tar.gz
Installed plugin: 2to3
$Next, migrate Helm v2 configuration in-place to Helm v3. This will migrate Chart starters, Repositories, and Plugins. The move config command will create the Helm v3 config and data folders if they don't exist, and will override the repositories.yaml file if it does exist.
Another helpful tip during this migration is to use ‘ — dry-run‘ option to test if everything looks okay.
hreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm 2to3 move config --help
migrate Helm v2 configuration in-place to Helm v3
Usage:
2to3 move config [flags]
Flags:
--dry-run simulate a command
-h, --help help for move
$shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm 2to3 move config --dry-run
.
.
. ..# Note: Not showing the full output
.
$shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm 2to3 move config
2020/03/12 07:24:50 WARNING: Helm v3 configuration may be overwritten during this operation.
2020/03/12 07:24:50
[Move Config/confirm] Are you sure you want to move the v2 configuration? [y/N]: y
2020/03/12 07:24:51
Helm v2 configuration will be moved to Helm v3 configuration.
2020/03/12 07:24:51 [Helm 2] Home directory: /home/shreyans_mulkutkar/.helm
2020/03/12 07:24:51 [Helm 3] Config directory: /home/shreyans_mulkutkar/.config/helm
2020/03/12 07:24:51 [Helm 3] Data directory: /home/shreyans_mulkutkar/.local/share/helm
2020/03/12 07:24:51 [Helm 3] Cache directory: /home/shreyans_mulkutkar/.cache/helm
2020/03/12 07:24:51 [Helm 3] Create config folder "/home/shreyans_mulkutkar/.config/helm" .
2020/03/12 07:24:51 [Helm 3] Config folder "/home/shreyans_mulkutkar/.config/helm" created.
2020/03/12 07:24:51 [Helm 2] repositories file "/home/shreyans_mulkutkar/.helm/repository/repositories.yaml" will copy to [Helm 3] config folder "/home/shreyans_mulkutkar/.config/helm/repositories.yaml" .
2020/03/12 07:24:51 [Helm 2] repositories file "/home/shreyans_mulkutkar/.helm/repository/repositories.yaml" copied successfully to [Helm 3] config folder "/home/shreyans_mulkutkar/.config/helm/repositories.yaml" .
2020/03/12 07:24:51 [Helm 3] Create cache folder "/home/shreyans_mulkutkar/.cache/helm" .
2020/03/12 07:24:51 [Helm 3] cache folder "/home/shreyans_mulkutkar/.cache/helm" created.
2020/03/12 07:24:51 [Helm 3] Create data folder "/home/shreyans_mulkutkar/.local/share/helm" .
2020/03/12 07:24:51 [Helm 3] data folder "/home/shreyans_mulkutkar/.local/share/helm" created.
2020/03/12 07:24:51 [Helm 2] starters "/home/shreyans_mulkutkar/.helm/starters" will copy to [Helm 3] data folder "/home/shreyans_mulkutkar/.local/share/helm/starters" .
2020/03/12 07:24:51 [Helm 2] starters "/home/shreyans_mulkutkar/.helm/starters" copied successfully to [Helm 3] data folder "/home/shreyans_mulkutkar/.local/share/helm/starters" .
2020/03/12 07:24:51 Helm v2 configuration was moved successfully to Helm v3 configuration.
$
After moving the configuration, Helm still points to version v2.16.3 (Remember, I had copied/renamed my original Helm v2 binary from ‘helm’ to ‘helm2’).
Also, the existing application release is unaffected and nothing shows up under Helm v3 release lists.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm2 version
Client: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm3 version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm3 list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm2 list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
kneeling-zebra 1 Thu Mar 12 07:16:08 2020 DEPLOYED helm_example-0.1.0 1.0 default
$Next, migrate existing Helm v2 releases in-place to Helm v3.
There is a limit set on the number of versions/revisions of a release that are converted. It is defaulted to 10 but can be configured with the --release-versions-max flag. When the limit set is less than the actual number of versions then only the latest release versions up to the limit will be converted. Older release versions with not be converted. If --delete-v2-releases is set, the older versions will remain in Helm v2 storage but will no longer be visible to Helm v2 commands like helm list. Clean up will remove them from storage.
Like mentioned for ‘helm 2to3 move config’ command, you can start with the dry-run option ‘helm 2to3 convert --dry-run kneeling-zebra’. If everything looks okay, proceed with the final migration.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm 2to3 convert kneeling-zebra
2020/03/12 07:27:32 Release "kneeling-zebra" will be converted from Helm v2 to Helm v3.
2020/03/12 07:27:32 [Helm 3] Release "kneeling-zebra" will be created.
2020/03/12 07:27:32 [Helm 3] ReleaseVersion "kneeling-zebra.v1" will be created.
2020/03/12 07:27:32 [Helm 3] ReleaseVersion "kneeling-zebra.v1" created.
2020/03/12 07:27:32 [Helm 3] Release "kneeling-zebra" created.
2020/03/12 07:27:32 Release "kneeling-zebra" was converted successfully from Helm v2 to Helm v3.
2020/03/12 07:27:32 Note: The v2 release information still remains and should be removed to avoid conflicts with the migrated v3 release.
2020/03/12 07:27:32 v2 release information should only be removed using `helm 2to3` cleanup and when all releases have been migrated over.
$Now you can see the ‘kneeling-zebra’ release also under Helm3 list. It also continues to show up under Helm2. This is because we haven’t cleaned up Helm v2 configuration, release data, and Tiller deployment.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm3 list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kneeling-zebra default 1 2020-03-12 14:16:08.168493614 +0000 UTC deployed helm_example-0.1.0 1.0
$shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm2 list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
kneeling-zebra 1 Thu Mar 12 07:16:08 2020 DEPLOYED helm_example-0.1.0 1.0 default
$
After the migration, note that the Tiller deployment and service required in v2 architecture still exists and our NGINX application Pods, Deployment, Service is unaffected.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ kubectl get deploy,svc tiller-deploy -n kube-system
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/tiller-deploy 1/1 1 1 17m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/tiller-deploy ClusterIP 10.51.240.136 <none> 44134/TCP 17m
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deploy-example-78c668d7fb-ffg6x 1/1 Running 0 15m
pod/nginx-deploy-example-78c668d7fb-gr7g9 1/1 Running 0 15m
pod/nginx-deploy-example-78c668d7fb-t4pxw 1/1 Running 0 15m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 29m
service/nginx-svc-example ClusterIP 10.51.245.227 <none> 80/TCP 15m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deploy-example 3/3 3 3 15m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deploy-example-78c668d7fb 3 3 3 15m
$
Finally, the time has arrived to clean up all Helm v2 data. ‘helm 2to3 cleanup’ command cleans up the v2 configuration (Helm home directory), v2 release data and the Tiller deployment. The command will ask for a confirmation on whether you really want to cleanup Helm v2 data.
Clean up can be done individually also, by setting one or all of the following flags: --config-cleanup, --release-cleanup and --tiller-cleanup. Cleanup of a release and its versions is done by setting --name flag. This is a singular operation and is not to be used with the other cleanup operations. If none of these flags are set, then all cleanup is performed.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm 2to3 cleanup
WARNING: "Helm v2 Configuration" "Release Data" "Release Data" will be removed.
This will clean up all releases managed by Helm v2. It will not be possible to restore them if you haven't made a backup of the releases.
Helm v2 may not be usable afterwards.
[Cleanup/confirm] Are you sure you want to cleanup Helm v2 data? [y/N]: y
2020/03/12 07:32:49
Helm v2 data will be cleaned up.
2020/03/12 07:32:49 [Helm 2] Releases will be deleted.
2020/03/12 07:32:49 [Helm 2] ReleaseVersion "kneeling-zebra.v1" will be deleted.
2020/03/12 07:32:49 [Helm 2] ReleaseVersion "kneeling-zebra.v1" deleted.
2020/03/12 07:32:49 [Helm 2] Releases deleted.
2020/03/12 07:32:49 [Helm 2] Tiller in "kube-system" namespace will be removed.
2020/03/12 07:32:49 [Helm 2] Tiller "deploy" in "kube-system" namespace will be removed.
2020/03/12 07:32:49 [Helm 2] Tiller "deploy" in "kube-system" namespace was removed successfully.
2020/03/12 07:32:49 [Helm 2] Tiller "service" in "kube-system" namespace will be removed.
2020/03/12 07:32:49 [Helm 2] Tiller "service" in "kube-system" namespace was removed successfully.
2020/03/12 07:32:49 [Helm 2] Tiller in "kube-system" namespace was removed.
2020/03/12 07:32:49 [Helm 2] Home folder "/home/shreyans_mulkutkar/.helm" will be deleted.
2020/03/12 07:32:49 [Helm 2] Home folder "/home/shreyans_mulkutkar/.helm" deleted.
2020/03/12 07:32:49 Helm v2 data was cleaned up successfully.
$After Helm v2 cleanup, the Tiller deployment no longer exists.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm2 version
Client: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
Error: could not find tiller
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ kubectl get deploy,svc tiller-deploy -n kube-system
Error from server (NotFound): deployments.extensions "tiller-deploy" not found
Error from server (NotFound): services "tiller-deploy" not found
$Also, you cannot see any releases under ‘helm2 list’ as everything got deleted when ‘helm 2to3 cleanup’ command was used.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm2 list
Error: could not find tiller
$Our NGINX application Pods are unaffected. This can be confirmed by looking at the ‘AGE’ of individual K8s objects.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deploy-example-78c668d7fb-ffg6x 1/1 Running 0 18m
pod/nginx-deploy-example-78c668d7fb-gr7g9 1/1 Running 0 18m
pod/nginx-deploy-example-78c668d7fb-t4pxw 1/1 Running 0 18m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.51.240.1 <none> 443/TCP 32m
service/nginx-svc-example ClusterIP 10.51.245.227 <none> 80/TCP 18m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deploy-example 3/3 3 3 18m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deploy-example-78c668d7fb 3 3 3 18m
$Finally, Helm3 takes over the release.
Note: After Helm2 is deleted and only Helm 3 manages K8s packages, Helm3 can still take v2 charts. That means, at this point, you could still make changes to your Helm v2 templates/values.yaml and upgrade using Helm3 binary.
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm3 version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
$
shreyans_mulkutkar@cloudshell:~/.helm (smulkutk-project-1)$ helm3 list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
kneeling-zebra default 1 2020-03-12 14:16:08.168493614 +0000 UTC deployed helm_example-0.1.0 1.0
$
