Skip to content

Storage Service

Storage provides both runtime key-value operations and internal configuration persistence.

Base URL examples:

  • Local: http://localhost:8086
  • Cluster: http://storage:8086

Capability Areas

  • Runtime key-value storage for workflow data.
  • Internal storage for workflows, connections, policies, and schedules.
  • Runtime config hierarchy (global, group, workflow scopes).
  • Secret upload/download and secret value retrieval.

Endpoint Groups

Health

  • GET /healthz
  • GET /readyz

Runtime key-value API

  • GET /v1/storage/{bucket}/{key}
  • PUT /v1/storage/{bucket}/{key}
  • GET /v1/storage
  • GET /v1/storage/{bucket}

Internal artifact APIs

  • /internal/workflows*
  • /internal/connections*
  • /internal/policies*
  • /internal/schedules*

Internal runtime config APIs

  • /internal/config/global*
  • /internal/config/groups*
  • /internal/config/workflows*
  • /internal/config/resolved

Internal secret APIs

  • /internal/upload/secrets (?overwrite=true default upsert; overwrite=false exclusive, 409 if any name exists)
  • /internal/download/secrets
  • /internal/secrets* (POST /internal/secrets/{name}?overwrite= same exclusive vs upsert semantics)

Internal upload APIs (single-item)

All upload endpoints require the runtime-admin token.

  • POST /internal/upload/workflows/{name} (YAML body)
  • POST /internal/upload/connections/{name} (YAML body)
  • POST /internal/upload/policies/{name} (YAML body)
  • POST /internal/upload/schedules/{name} (YAML body)
  • POST /internal/upload/config (YAML body, scoped runtime settings)

After uploading a workflow, trigger a reload in the orchestrator to pick up the new definition (POST /admin/reload).

Usage Notes

  • Internal endpoints are intended for trusted services and operational tooling.
  • Runtime config resolution is used by Orchestrator for merged effective config.
  • Secret data should only be accessed by least-privileged service identities.

Runtime provisioning

Storage does not seed or migrate settings, secrets, connections, workflows, schedules, or policies. External installs boot with an empty runtime store.

Provision assets explicitly via:

  • Dashboard (Secrets, Connections, Workflows, …)
  • POST /internal/upload/* and POST /internal/upload/config
  • POST /internal/bootstrap/from-yaml (directory-based import)
  • Deploy-time sync (for lab/CI only), including dummy runtime secrets example-dummy-overwrite (upsert) and example-dummy-exclusive (create-once)

Optional example YAML under flows/ is for manual upload only — nothing is applied automatically by the storage service.

Operational Guidance

  • Keep backups/snapshots for persistent storage data.
  • Restrict network access to internal routes.
  • Monitor latency and error rates on key-value operations.
  • Use explicit bucket/key naming conventions per domain.