Helm Charts in Production — Templating, Testing, and Chart Promotion Strategies

Introduction

Helm is the package manager for Kubernetes, but a poorly designed chart is worse than no chart at all — hard-coded values, missing security contexts, and untested templates create a maintenance nightmare that diverges across environments until something fails silently in production. Well-designed charts have production-safe defaults, named template helpers, environment-specific value overrides, pre-upgrade hooks for migrations, and unit tests that catch regressions before deployment. This post covers everything from chart structure to multi-chart orchestration with Helmfile.

Chart Structure and Production-Safe Defaults

A production Helm chart follows a consistent directory layout and defaults that are secure and sensible without additional configuration:

my-app-chart/
├── Chart.yaml
├── values.yaml          # Secure defaults — production-safe out of the box
├── values-staging.yaml  # Staging overrides
├── values-prod.yaml     # Production overrides
├── templates/
│   ├── _helpers.tpl     # Named template fragments
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── hpa.yaml
│   ├── pdb.yaml
│   └── tests/
│       └── test-connection.yaml
└── charts/              # Bundled dependencies

The values.yaml should be deployable without modification and enforce security best practices:

replicaCount: 3
 
image:
  repository: my-app
  pullPolicy: IfNotPresent
  tag: "2.3.1"
 
podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 1000
 
securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop:
    - ALL
 
resources:
  limits:
    cpu: 1000m
    memory: 512Mi
  requests:
    cpu: 250m
    memory: 256Mi
 
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 75
 
livenessProbe:
  httpGet:
    path: /healthz
    port: http
  initialDelaySeconds: 30
  periodSeconds: 10
  failureThreshold: 3
 
readinessProbe:
  httpGet:
    path: /ready
    port: http
  initialDelaySeconds: 10
  periodSeconds: 5
  failureThreshold: 2

New teams deploying this chart get non-root execution, read-only filesystem, dropped capabilities, resource limits, and HPA out of the box. Security is the default, not an option.

Named Templates and Helpers

Reusable template fragments belong in _helpers.tpl. This eliminates duplication and ensures label consistency across all resources:

{{- define "my-app.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
 
{{- define "my-app.labels" -}}
helm.sh/chart: {{ printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
app.kubernetes.io/name: {{ default .Chart.Name .Values.nameOverride }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
 
{{- define "my-app.selectorLabels" -}}
app.kubernetes.io/name: {{ default .Chart.Name .Values.nameOverride }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

The deployment template references these helpers rather than duplicating label logic:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-app.fullname" . }}
  labels:
    {{- include "my-app.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "my-app.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      labels:
        {{- include "my-app.selectorLabels" . | nindent 8 }}
    spec:
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
      - name: {{ .Chart.Name }}
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
        imagePullPolicy: {{ .Values.image.pullPolicy }}
        securityContext:
          {{- toYaml .Values.securityContext | nindent 12 }}
        resources:
          {{- toYaml .Values.resources | nindent 12 }}
        livenessProbe:
          {{- toYaml .Values.livenessProbe | nindent 12 }}
        readinessProbe:
          {{- toYaml .Values.readinessProbe | nindent 12 }}

Pre-Upgrade Hooks for Database Migrations

Helm hooks execute Jobs at defined lifecycle points. Database migrations should run as a pre-upgrade hook before the new application version receives traffic:

apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "my-app.fullname" . }}-db-migrate
  labels:
    {{- include "my-app.labels" . | nindent 4 }}
  annotations:
    helm.sh/hook: pre-upgrade
    helm.sh/hook-weight: "0"
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      serviceAccountName: {{ include "my-app.fullname" . }}
      containers:
      - name: migrate
        image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
        command: ["/app/migrate.sh"]
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: {{ include "my-app.fullname" . }}-db
              key: url

Hook weight controls execution order within the same hook type. Negative weights run first; the migration hook at weight 0 runs before application startup at weight 5.

Testing with helm-unittest

helm-unittest validates rendered manifests without deploying to a cluster:

# templates/tests/deployment_test.yaml
suite: test deployment
templates:
  - deployment.yaml
tests:
  - it: should render with correct replica count
    asserts:
      - equal:
          path: spec.replicas
          value: 3
 
  - it: should enforce non-root execution
    asserts:
      - equal:
          path: spec.template.spec.securityContext.runAsNonRoot
          value: true
      - equal:
          path: spec.template.spec.containers[0].securityContext.allowPrivilegeEscalation
          value: false
 
  - it: should not set replicas when autoscaling is enabled
    set:
      autoscaling.enabled: true
    asserts:
      - notExists:
          path: spec.replicas
 
  - it: should use custom replica count when autoscaling disabled
    set:
      autoscaling.enabled: false
      replicaCount: 7
    asserts:
      - equal:
          path: spec.replicas
          value: 7

Run tests with helm unittest ./my-app-chart. These tests belong in CI and run on every chart change before any cluster is touched.

Environment Promotion with Helmfile

Helmfile manages multiple charts with environment-specific values and deployment ordering:

# helmfile.yaml
helmDefaults:
  atomic: true
  wait: true
  timeout: 600
 
repositories:
- name: bitnami
  url: https://charts.bitnami.com/bitnami
 
releases:
  - name: postgres
    namespace: databases
    chart: bitnami/postgresql
    version: "14.x"
    values:
    - ./values-postgres.yaml
 
  - name: my-app
    namespace: production
    chart: ./my-app-chart
    version: 1.5.2
    values:
    - ./values.yaml
    - ./values-prod.yaml
    needs:
    - databases/postgres

Production overrides in values-prod.yaml set appropriate scale and resource limits without duplicating the entire values file:

# values-prod.yaml — only what differs from defaults
replicaCount: 5
autoscaling:
  minReplicas: 5
  maxReplicas: 50
resources:
  limits:
    cpu: 1500m
    memory: 1Gi
  requests:
    cpu: 500m
    memory: 512Mi

Deploy with helmfile -f helmfile.yaml apply. The needs field ensures postgres is healthy before the application chart deploys.

Chart Versioning and OCI Registry

Store chart artifacts in an OCI registry for immutable versioned releases:

# Package the chart
helm package ./my-app-chart
 
# Push to OCI registry
helm push my-app-chart-1.5.2.tgz oci://registry.example.com/helm
 
# Deploy a specific pinned version
helm upgrade --install my-app \
  oci://registry.example.com/helm/my-app \
  --version 1.5.2 \
  -f values-prod.yaml

Pin exact versions in production — never use floating latest tags. The chart version in Chart.yaml should follow semver and increment on any template change, even if the application version stays the same.

Key Takeaways

values.yaml base defaults must be production-safe — non-root execution, read-only filesystem, resource limits, and dropped capabilities should be on by default
Named template helpers in _helpers.tpl eliminate duplication and ensure consistent labeling across all Kubernetes resources
Pre-upgrade hooks for database migrations guarantee schema changes run before the new application version receives traffic
Unit tests with helm-unittest validate rendered manifests in CI before any cluster deployment
Helmfile manages multi-chart deployments with needs ordering, environment-specific values, and atomic rollbacks
Store chart artifacts in an OCI registry with pinned semver versions — floating tags in production cause undocumented environment drift
Run helm lint and helm template in CI to catch syntax errors before they reach a cluster
Separate secrets from values — use helm-secrets or external secret operators rather than embedding sensitive values in values files

Introduction#

Chart Structure and Production-Safe Defaults#

Named Templates and Helpers#

Pre-Upgrade Hooks for Database Migrations#

Testing with helm-unittest#

Environment Promotion with Helmfile#

Chart Versioning and OCI Registry#

Key Takeaways#

Related reading