Skip to content

Kubernetes Deployment

Deploy Engine services to Kubernetes. CI is the primary path for the shared Linux cluster — pushes and merges trigger Forgejo Actions (build.yml → deploy.yml). Use this guide for context, manual bootstrap, and multi-environment layout.

See also: Develop and Test, CI/CD Pipeline, Manual K8s Deployment.

Multi-Environment Layout

All environments share one cluster with isolated namespaces and NodePorts:

Environment Namespace HTTP HTTPS
lab meshflows-lab 31100 31101
dev meshflows-dev 31200 31201
tst meshflows-tst 31300 31301
acc meshflows-acc 31400 31401
prd meshflows-prd 31500 31501

WAN traffic enters via edge-router (meshflows-edge) on 30080/30443; per-env nginx Gateway Fabric handles TLS and HTTPRoutes.

CI deploy uses engine/deploy/overlays/linux-cluster/ with release-bundle image tags (vYYYY.MM.DD.<run>). Legacy per-env overlays under engine/deploy/k8s/overlays/{dev,tst,...} use static tags (:dev, :tst) for manual experiments only.

Automatic promotion:

  • Feature branches / PRs → dev
  • Merge to main → dev, then tst + lab
  • flows/ only push to main → tst (no engine rebuild)

Manual promotion to lab/acc/prd: Forgejo Actions → Deploy to Cluster with release_id from build artifact or cluster release index.

Manual Deploy (Bootstrap / Lab)

Linux cluster overlay

cd engine/deploy
kubectl apply -k overlays/linux-cluster/

Legacy K8s overlays (local experiments)

cd engine/deploy
kubectl apply -k overlays/minikube/    # local minikube (preferred; avoids kustomize cycle bug)
kubectl apply -k k8s/overlays/dev/      # namespace meshflows-dev (legacy experiments)
kubectl apply -k k8s/overlays/tst/      # namespace meshflows-tst

Verify Rollout

kubectl get pods -n meshflows-dev
kubectl get svc -n meshflows-dev
kubectl rollout status deployment/orchestrator -n meshflows-dev

Replace namespace with your target environment.

Sync Flows Data PV

Sync flows content into the shared PVC mounted at /app/flows:

cd engine
./scripts/sync-flows-pv.sh meshflows-dev

Argument is the namespace prefix (e.g. meshflows-dev, not meshflows alone).

Required Inputs

  • Namespace and image pull secret (forgejo-registry).
  • Runtime secrets (JWT, bootstrap credentials, RabbitMQ, service tokens).
  • Storage class/PVC availability (Longhorn recommended).
  • Forgejo environment secrets for CI deploy — see Secrets Management.

Additional Hostnames: Code and Webmail

The deploy workflow also manages:

  • code.meshflows.org → Forgejo (namespace forgejo)
  • webmail.meshflows.org → Roundcube (namespace webmail)

Manifests live under engine/deploy/overlays/linux-cluster/infra/. Roundcube values can be set via Forgejo secrets (ROUNDCUBE_*).

Post-Deploy Checks

  • Gateway status endpoint responds.
  • Orchestrator readyz reports loaded workflows/connections/policies.
  • RabbitMQ and storage components are healthy.
  • Workflow/schedule/config files resolve from /app/flows/*.

Upgrade Pattern

Via CI: merge to main or run manual deploy.yml with a known release_id.

Manual:

  1. Build and push images (or pull published tags from Forgejo packages).
  2. Update image tags via deploy_engine_images.py or overlay patches.
  3. Apply manifests.
  4. Monitor rollout and traces.

Rollback: redeploy a previous release_id or use deployment rollout history.

TLS and Gateway

HTTPS uses nginx Gateway Fabric + cert-manager. See Lab HTTPS Setup and run infra.yml for cluster infrastructure.