Skip to content

Egress S3 (MinIO / S3-compatible)

Responsibility

Object-store capability provider for S3-compatible backends (in-cluster MinIO first). Workflows call it via service_call — no native orchestrator storage steps.

Capabilities:

  • egress.s3.put — put object. Designer fields: bucket, key, body / body_base64, content_type, metadata (no JSON envelope required)
  • egress.s3.get — get object (bucket, key, optional as_base64)
  • egress.s3.list — list objects (bucket, prefix, limit)
  • egress.s3.delete — delete object (bucket, key)
  • egress.s3.trigger.object — start-contract trigger (invocation.type: s3)
  • s3.connection — connection type s3 (endpoint, access/secret key, region, secure, default_bucket)

Admin routes (dashboard proxy, X-S3-Admin-Token):

  • GET /admin/buckets, POST /admin/buckets
  • GET /admin/objects, GET|PUT|DELETE /admin/object

Architecture

Capability provider (see Deploy integration contract):

  • Registers with capability-registry at startup
  • Default endpoint: http://orchestrator-minio:9000
  • Bootstraps bucket object-demo when default credentials are set
  • Distinct from the platform storage service (filesystem KV / runtime docs under /app/data)

Connection Model

name: minio_demo
type: s3
endpoint: http://orchestrator-minio:9000
region: us-east-1
secure: false
default_bucket: object-demo

Reference: flows/connections/minio_demo.yaml.

In-cluster demo connection uses runtime secrets (same store as Dashboard → Secrets):

access_key: $secret:minio-access-key
secret_key: $secret:minio-secret-key

Deploy syncs those from K8s secret meshflows-orchestrator-minio into Storage (/internal/secrets, which the dashboard proxies). The egress-s3 pod can still fall back to env defaults from that K8s secret when a connection omits keys.

Example Workflow

  • Workflow: flows/workflows/storage_minio_demo.yaml
  • HTTP trigger: POST /v1/api/integration/storage-minio-demo
  • Object eventstream trigger: flows/triggers/storage-minio-object-trigger.yaml (invocation.type: s3) → storage_minio_object_trigger_demo. Subscriber s3-trigger-subscriber listens to MinIO listen_bucket_notification (eventstream; polling only if the backend has no notification API). Filters: create/update of .xml/.json under inbox/. Delete events include snapshot_key under .meshflows-snapshots/ so a late consumer can still GET the last copy.
  • DSAV archive: flows/workflows/dsav_message_archive.yaml + AMQP trigger flows/triggers/dsav-message-archive-amqp.yaml — stores each DSAV message at Domain/Service/Action/Version/YYYY/MM/DD/<request_id>.json on MinIO (requires SES for the javascript key-builder step).

Deploy

  • Image: meshflows/egress-s3
  • Deployment: egress-s3
  • Trigger worker: s3-trigger-subscriber (same image, S3_TRIGGER_ENABLED=true)
  • Service path: engine/services/s3/
  • Secret: meshflows-orchestrator-minio (access_key / secret_key)

Health

  • GET /healthz
  • GET /readyz
  • GET /internal/capabilities
  • Cluster health maps s3.service → deployment egress-s3
  • Cluster health maps s3-trigger-subscriber → deployment s3-trigger-subscriber