Skip to content

Step Contract (Fields, Options, Rules)

This page documents the step-level contract for workflow YAML.

Scope: - Field-level options per step type. - Required vs optional fields. - Common validation and behavior rules.

Source of truth: - engine/services/orchestrator/app/workflow_runner.py - WorkflowStep discriminated union and corresponding step models.

Terminology

A capability provider is an HTTP service that registers with the capability-registry. Workflows use exclusively type: service_call plus capability_name and path for integrations, transforms, and scripts. Built-in control/context steps remain orchestrator-owned. Do not add new first-class integration types.

Shared Contract (Most Step Types)

Common fields used by many steps through shared mixins:

Field Required Type Description
id yes string Unique step identifier within the workflow.
type yes string Step discriminator (for example service_call, for_each).
when no object Conditional execution rule based on context values.
run_after no list Dependency gating based on previous step status.
compensate no object Compensation steps for rollback behavior.
retry_policy no object Per-step retry override (where supported).

Notes: - Use id:, not name:, for step identity. - when supports equals, not_equals, or one_of (exactly one). - run_after supports concise YAML {step_id: succeeded} and a status list {step_id: [succeeded, failed]} (OR within one predecessor, AND across predecessors). - Status values: succeeded, failed, skipped, timedout. Combinations match Logic Apps Configure run after. - Control-flow steps may handle retries differently than service_call.

Step Type Registry

Supported values for type (exact WorkflowStep union):

  • context_set
  • context_increment
  • context_bind
  • context_extract_json
  • context_extract_xml
  • milestone
  • json_set
  • xml_set_text
  • terminate
  • wait
  • log
  • workflow_call
  • collect
  • for_each
  • repeat_until
  • if
  • try
  • scope
  • parallel
  • service_call

Former first-class integration/transform/script types (http, xslt, ftp, storage_write, …) are not in the union. Map them via service_call — see Step Types.

Contract by Step Type

service_call

Required: - id - type: service_call - capability_name - path

Optional: - capability_version (recommended, e.g. "^1.0") - connection - payload / payload_from - first-class provider fields (capability-specific; see below) - output_key - timeout_seconds - values (placeholder substitution) - variable-binding fields (variable_from, operand, apply, result_to) for variables.* - retry_policy - shared fields

Validation rules: - Do not mix exclusive payload modes (see model validators in workflow_runner.py). - Capability must be discoverable via the capability registry at runtime.

Common capability contracts

script.execute

Required first-class fields: runtime, code. Path: /v1/execute.

Optional: payload_from (SES input), timeout_ms, legacy JSON payload.

See Script steps howto.

For C# / .NET custom logic, use a capability provider + service_call (not SES): C# Custom Process.

egress.http.call

Path: /call. First-class: url (or legacy payload.url), optional method, headers, body, request_timeout_seconds, fail_on_error_status, accepted_status_codes, values.

egress.rabbitmq.topic.send

Path: /publish/topic. First-class: domain, service, action, version; optional content_type, message, payload_from.

egress.smtp.call

Path: /call. First-class: to_emails, subject, body / body_from, is_html, attachments, …

egress.imap.call

Path: /call. First-class: action (peek | list | read), mailbox, filter, limit, optional message_id, include_body, include_attachments, mark_seen, attachment_max_bytes.

action Provider behaviour
peek Headers of matches without setting \Seen
list Compact summary list
read Full message(s); honour include_* / mark_seen

See Egress IMAP. Legacy JSON payload remains a compose fallback.

egress.rabbitmq.publish

Path: /publish. Generic publish (exchange/queue routing in payload/connection).

Transforms (transformers.*)

Capability Path
transformers.apply_xslt /applyXSLT
transformers.xml2json /xml2json
transformers.json2xml /json2xml
transformers.apply_liquid /applyLiquid
transformers.csv_to_json /csv2json
transformers.json_to_csv /json2csv
transformers.xml_to_csv /xml2csv
transformers.csv_to_xml /csv2xml
transformers.flatfile_to_xml /flatfile2xml
transformers.xml_to_flatfile /xml2flatfile
transformers.validate_json_schema /validateJsonSchema
transformers.validate_xsd /validateXsd

Demo: flows/workflows/transform_testing_demo.yaml, flows/workflows/csv_transform_demo.yaml, flows/workflows/xml_csv_demo.yaml, flows/workflows/flatfile_demo.yaml, flows/workflows/json_schema_validate_demo.yaml, flows/workflows/xsd_validate_demo.yaml, flows/workflows/soap_http_xslt_demo.yaml.

Object storage (egress.s3.*)

Capability Path
egress.s3.put /put
egress.s3.get /get
egress.s3.list /list
egress.s3.delete /delete

There are no native storage_read / storage_write steps. Demo: flows/workflows/storage_minio_demo.yaml.

Other frequent capabilities

Capability Path (typical)
ftp.call /ftp
egress.ssh.exec /exec
egress.ssh.sftp /sftp
sftp.list / download / upload /sftp/list, /sftp/download, /sftp/upload
egress.imap.call /call
egress.smtp.call /call
egress.google_workspace.* e.g. /gmail/list
variables.string_concat / number_add / … /variables/...

Naming note: not every capability uses the egress. prefix.

context_set

Required: - id - type: context_set - context_key - exactly one of value or value_from

Optional: - retry_policy - shared fields

Validation rules: - Fails validation if both value and value_from are set, or both missing.

milestone

Required: - id - type: milestone - exactly one of value or value_from

Optional: - retry_policy - shared fields

Writes context.milestone and context.milestone_at (UTC). Operations list-API rolls the latest written value in the related-run tree onto the runtime-root row.

Validation rules: - Fails validation if both value and value_from are set, or both missing.

context_increment

Required: - id - type: context_increment - context_key

Optional: - by (default 1) - shared fields

context_bind

Parse JSON, XML, or YAML once and register it under an alias for path references (request:json:/field). See Bound documents.

Required:

  • id
  • type: context_bind
  • alias

Optional:

  • input_from
  • format (auto, json, xml, yaml)
  • shared fields

context_extract_json

Required: - id - type: context_extract_json - context_key - json_path

Optional: - input_from - retry_policy - shared fields

context_extract_xml

Required: - id - type: context_extract_xml - context_key - xpath

Optional: - input_from - shared fields

json_set

Required: - id - type: json_set - json_path - value_from

Optional: - input_from - mirror_to_context - shared fields

xml_set_text

Required: - id - type: xml_set_text - xpath - value_from

Optional: - input_from - attribute - mirror_to_context - shared fields

terminate

Required: - id - type: terminate - status in: success, failed, ignored, canceled, skipped

Optional: - reason - http_status (range 200..599) - shared fields

Behavior: - Stops workflow execution immediately and returns a controlled termination state.

wait

Required: - id - type: wait - seconds (range 0..3600)

Optional: - shared fields

Behavior: - Pauses execution for the configured duration.

log

Required: - id - type: log - message

Optional: - values (default {}) - level (debug, info, warning, error; default info) - tags (default []) - into - shared fields

Behavior: - Writes structured log event and can store rendered message in context via into. - Tag business-event marks a business event (Loki marker meshflows.business_event). See Business events.

workflow_call

Required: - id - type: workflow_call - target

Optional: - payload - output_key - timeout_seconds (range 1..3600) - shared fields

Behavior: - Invokes another workflow as a child run. - Applies cycle detection and maximum call depth. - Inherits parent timeout budget, optionally capped by timeout_seconds. - Emits trace linkage metadata (parent_run_id, child_run_id, depth/call-chain context).

collect

Required: - id - type: collect - from_steps (at least one entry)

Optional: - merge_mode (object or array, default object) - try_parse_json (default true) - shared fields

Notes: - from_steps entries can be: - step_id - {step_id: alias}

Validation rules: - Mapping entries in from_steps must contain exactly one key.

for_each

Required: - id - type: for_each - steps (at least one nested step)

Optional: - input_from - items_path (default /) - as_key (default item) - index_as - max_iterations (default 100, range 1..10000) - shared fields

Behavior: - Iterates over array items and executes nested steps per item.

repeat_until

Required: - id - type: repeat_until - until - steps (at least one nested step)

Optional: - max_iterations (default 20, range 1..10000) - shared fields

Behavior: - Repeats nested steps until condition matches or limit is reached.

if

Required: - id - type: if - condition - then (or then_steps)

Optional: - else (or else_steps, default empty) - shared fields

condition is a WhenCondition object:

Field Required Description
context_key (alias variable) yes Run context key to read
equals exactly one of Expected string value
not_equals these three Must not equal
one_of Value must be in list

Behavior: - Executes then branch when condition matches, otherwise else. - Compares context strings only — no JSON Pointer or XPath in context_key. - To branch on a field inside trigger JSON, XML, or service_call output, use context_extract_json / context_extract_xml first (see Conditions & Path Expressions).

try

Required: - id - type: try - steps (at least one nested step)

Optional: - catch (or catch_steps, default empty) - rethrow (default false) - error_context_key (default {step_id}.error) - shared fields

Behavior: - Runs steps sequentially; on failure stores the error message in context and runs catch substeps when present. - When catch is empty, failures propagate like a normal step failure. - When rethrow is false (default), the try step is marked successful after catch and the workflow continues. - When rethrow is true, catch substeps still run but the original error is raised afterward (workflow fails). - WorkflowTerminatedByStep is never caught and always stops the run immediately. - Failed service_call steps that received an HTTP error envelope (e.g. egress fail_on_error_status) still write that JSON to outputs[<step_id>] and context:<step_id>.response, so catch can use input_from: <step_id> / extract /body / /headers / /status_code.

scope

Required: - id - type: scope - steps (at least one nested step)

Optional: - shared fields (when, run_after, compensate)

Behavior: - Runs steps sequentially in the same run context as the parent (sub-workflow grouping). - Output of the scope step is the output of the last inner step. - Inner failures propagate; use type: try when you need catch. - Distinct from parallel.scopes (named concurrent lanes) and from scoped_context (workflow variables).

Workflow-level on_error / finally

Optional lists on the workflow document (not a step type):

steps:
  - id: main
    type: log
    message: "work"
on_error:
  - id: handle
    type: log
    message: "failed: {err}"
    values:
      err: context:workflow.error
finally:
  - id: always
    type: log
    message: "cleanup"
  • on_error runs after the main run_after graph if a step failure is still pending. Sets context:workflow.error.
  • on_error_rethrow (default true): raise the original error after the handler. false swallows so the run succeeds.
  • finally always runs after main (and on_error), before return or re-raise.
  • Distinct from try/catch (local recovery on one step) and from run_after (sibling graph).

parallel

Required: - id - type: parallel - scopes (at least one named scope; each scope has nested steps)

Optional: - max_concurrency (default 10, range 1..100) - outputs_map (auto-generated from scope names if omitted) - fail_fast (default true) - shared fields

Validation rules: - Flat steps: on parallel is not supported — use scopes: only. - Do not set both scopes and legacy steps. - outputs_map values must be scope:<name> or step:<id> targeting scopes/steps that exist.

Contract Tips

  • Prefer connection-based service_call steps over inline credentials.
  • Keep step IDs stable; IDs are used by references (payload_from, run_after, collect, input_from).
  • Use workflow_call for composition, but avoid deep nesting unless required.
  • Validate workflows locally with:
python engine/scripts/validate-workflow.py flows/workflows/<your-file>.yaml

Or via orchestrator: POST /admin/validate?strict=true.