Skip to content

AKS Deployment

Deploy MeshFlows Engine to Azure Kubernetes Service (AKS) with the Kustomize overlay engine/deploy/overlays/aks/.

This path is for a standalone MeshFlows namespace on your own AKS cluster. It does not deploy the shared lab topology (Forgejo, webmail, Longhorn, multi-env NodePorts).

See also: Manual K8s Deployment (Path B), Kubernetes Deployment, overlay README.

What the overlay does

Item Behaviour
Base Full stack from engine/deploy/k8s/
Images code.meshflows.org/meshflowsgpl/meshflows/<service>
Namespace meshflows
Storage All PVC storageClassName → managed-csi (Azure Disk)
Extra PVCs tempo-data, orchestrator-traces, orchestrator-trigger-events
Ingress Gateway API (gatewayClassName: nginx) from base ingress.yaml

Day-0 checklist

Complete these before kubectl apply:

  1. Cluster — AKS with enough node capacity (lab starting point: 3× Standard_D4s_v5 or similar).
  2. Networking — Prefer Azure CNI Overlay (+ Cilium dataplane). Treat as hard to change later.
  3. StorageClass — Confirm managed-csi exists (kubectl get sc). For Premium SSD, change the overlay patch to managed-csi-premium.
  4. Gateway API — Install CRDs + a controller that registers GatewayClass nginx (nginx Gateway Fabric), or patch gatewayClassName.
  5. TLS — Install cert-manager and a ClusterIssuer named letsencrypt-prod, or enable patches/gateway-http-only.yaml for HTTP-only lab access.
  6. kubectl — az aks get-credentials for the target cluster.
  7. Secrets — At least meshflows-identity-auth and meshflows-orchestrator-auth (see below).
  8. Registry — Cluster can pull from Forgejo packages (or ACR mirror + imagePullSecret).

Create cluster (example)

export RG=meshflows-aks-rg
export LOC=westeurope
export CLUSTER=meshflows-aks

az group create -n "$RG" -l "$LOC"

az aks create \
  -g "$RG" -n "$CLUSTER" \
  --node-count 3 \
  --node-vm-size Standard_D4s_v5 \
  --network-plugin azure \
  --network-plugin-mode overlay \
  --network-dataplane cilium \
  --generate-ssh-keys

az aks get-credentials -g "$RG" -n "$CLUSTER"
kubectl get nodes
kubectl get sc

Use AKS Automatic instead of Standard when that matches your org defaults; the MeshFlows overlay only requires a compatible StorageClass and Gateway controller.

Install ingress prerequisites

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml

Install nginx Gateway Fabric so kubectl get gatewayclass shows nginx.

Install cert-manager and create a ClusterIssuer named letsencrypt-prod (same name as in base Gateway annotations), unless you use the HTTP-only patch.

Example ClusterIssuer (replace email):

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: you@example.com
    privateKeySecretRef:
      name: letsencrypt-prod-account
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: meshflows
                namespace: meshflows
                kind: Gateway

Solver details depend on your cert-manager / Gateway Fabric versions — follow current cert-manager Gateway API docs if the snippet above does not match your release.

Namespace and required secrets

export NAMESPACE=meshflows
kubectl create namespace "$NAMESPACE"

kubectl create secret generic meshflows-identity-auth \
  -n "$NAMESPACE" \
  --from-literal=jwt_secret="$(openssl rand -hex 32)" \
  --from-literal=encryption_key="$(openssl rand -hex 32)" \
  --from-literal=bootstrap_admin_user="admin" \
  --from-literal=bootstrap_admin_password="<strong-password>"

kubectl create secret generic meshflows-orchestrator-auth \
  -n "$NAMESPACE" \
  --from-literal=reload_token="$(openssl rand -hex 24)"

Full secret list: Secrets Management.

Optional: image pull secret

kubectl create secret docker-registry forgejo-registry \
  -n "$NAMESPACE" \
  --docker-server=code.meshflows.org \
  --docker-username=<user> \
  --docker-password=<token-or-password>

Then enable engine/deploy/overlays/aks/patches/default-sa-imagepullsecret.yaml in the overlay kustomization.yaml.

Deploy

cd engine/deploy
kubectl apply -k overlays/aks/ --load-restrictor=LoadRestrictionsNone

The --load-restrictor flag is required because ConfigMap generators reference files under flows/ outside deploy/k8s/ (same as minikube / linux-cluster).

Pin release tags in overlays/aks/kustomization.yaml (newTag) for production-like environments.

HTTP-only lab (no cert-manager)

Uncomment / add under patches: in the overlay:

patches:
  - path: patches/gateway-http-only.yaml
    target:
      kind: Gateway
      name: meshflows

(Keep the existing PVC storageClass patch.)

Verify

kubectl get pods -n meshflows
kubectl get pvc -n meshflows
kubectl get gateway,httproute -n meshflows
kubectl rollout status deployment/orchestrator -n meshflows
kubectl rollout status deployment/gateway -n meshflows

Sync flows into the shared PVC:

cd engine
./scripts/sync-flows-pv.sh meshflows

Temporary access without public DNS:

kubectl port-forward -n meshflows svc/gateway 8080:8080

Premium disks

Edit the PVC patch in overlays/aks/kustomization.yaml:

value: managed-csi-premium

What this path does not include

  • Longhorn / Contabo backups (lab linux-cluster)
  • Forgejo, Roundcube, edge-router multi-env NodePorts
  • Managed Azure Postgres / Redis / Blob (in-cluster defaults remain)
  • Azure DevOps / ACR promotion pipelines

Those can be added later without replacing this overlay as the MeshFlows runtime base.

Optional: Azure Workload Identity

To let egress-azure-servicebus (and the trigger subscriber) use a user-assigned managed identity instead of a SAS connection string:

  1. Enable OIDC issuer + the Azure Workload Identity mutating webhook on the cluster.
  2. Create a UAMI, federate it to ServiceAccount meshflows-azure-workload-identity.
  3. Apply engine/deploy/k8s/azure-workload-identity.example.yaml (replace CLIENT_ID) and/or enable overlays/aks/patches/azure-workload-identity.yaml.
  4. Set AZURE_SERVICEBUS_AUTH=workload_identity and AZURE_SERVICEBUS_FULLY_QUALIFIED_NAMESPACE on those deployments, or set auth / fully_qualified_namespace on the connection document.

See ADR 0012. SAS remains the default.