Skip to content

Add a Capability Provider

Extend MeshFlows without changing the Orchestrator. Register a small HTTP service with the capability registry; workflows call it through service_call steps. The Dashboard designer picks up metadata.designer for forms and labels.

This page is the starter kit: a minimal provider in Python, Go, and C#, plus the matching workflow YAML. For the full happy path (connection type, secrets, trigger, deploy checklist), see Integration end-to-end. For field-level rules, see the Capability provider contract. For Kubernetes / CI wiring, see Deploy integration.

Canonical Engine providers (Weatherstack, HTTP egress, …) are written in Python/FastAPI — use that stack in production MeshFlows images. Go and C# examples show that any language that speaks HTTP can implement the same contract.

Shipped demos (prefer these over pasting the snippets below when learning):

Language Service Capability Workflow / HTTP
C# engine/services/example-echo-csharp/ example.echo.csharp example_echo_csharp_demo → POST /v1/api/examples/echo-csharp
Go engine/services/example-echo-go/ example.echo.go example_echo_go_demo → POST /v1/api/examples/echo-go

The echo sample has no credentials. Real egress providers also register a workflow_connection capability and ship flows/connections/<type>.yaml — see Integration end-to-end and Egress Weatherstack.


What you build

Piece Purpose
POST /v1/registry/services/register Announce service_name, service_url, and capabilities[]
Heartbeat POST /v1/registry/services/{service_id}/heartbeat about every LEASE_TTL/2 seconds; re-register on 404/409
Capability HTTP API The path workflows call (here: POST /echo)
GET /internal/capabilities Local copy of the registration payload (ops / debugging)
Workflow step type: service_call + capability_name + path

Optional but recommended: Authorization: Bearer <CAPABILITY_REGISTRY_TOKEN> when the registry requires a token.


Shared registration payload

All three language samples register the same capability:

{
  "service_name": "example.echo",
  "service_type": "provider",
  "service_url": "http://example-echo:8080",
  "description": "Minimal echo capability provider.",
  "icon_emoji": "🔁",
  "capabilities": [
    {
      "name": "example.echo",
      "version": "1.0.0",
      "kind": "workflow_step",
      "description": "Echo a message back to the workflow.",
      "metadata": {
        "public_paths": [
          { "path": "/echo", "method": "POST", "auth": "none", "max_payload_bytes": 1048576 }
        ],
        "designer": {
          "display_name": "Echo message",
          "category": "examples",
          "step_type": "service_call",
          "icon_emoji": "🔁",
          "description": "Returns the input message — useful as a capability hello-world.",
          "form_fields": [
            {
              "name": "capability_name",
              "label": "Capability name",
              "type": "text",
              "required": true,
              "default": "example.echo",
              "yaml_path": "capability_name"
            },
            {
              "name": "path",
              "label": "Path on provider",
              "type": "text",
              "default": "/echo",
              "yaml_path": "path"
            },
            {
              "name": "message",
              "label": "Message",
              "type": "text",
              "required": true,
              "yaml_path": "message",
              "placeholder": "hello from MeshFlows"
            }
          ]
        }
      }
    }
  ]
}

Set service_url to the URL other cluster services use to reach your provider (Kubernetes service DNS in-cluster).


Environment

Variable Example Notes
SERVICE_URL http://example-echo:8080 Advertised base URL in the registry (in-cluster DNS, not localhost)
REGISTRY_URL or CAPABILITY_REGISTRY_URL http://capability-registry:8080 In-cluster / Compose service DNS (container listens on 8080)
CAPABILITY_REGISTRY_TOKEN (secret) Optional Bearer token
LEASE_TTL_SECONDS 90 Match registry lease; heartbeat ≈ half

Compose publishes the registry on the host as localhost:8093 → container 8080. Use http://localhost:8093 only from the host (curl/browser). Providers and Orchestrator inside the Compose network must use http://capability-registry:8080.


Implementation examples

Choose a language. All three samples implement the same example.echo contract (register, heartbeat, POST /echo).

Canonical Engine providers ship as Python / FastAPI; Go and C# show that any HTTP service can follow the contract.

Matches the style used by Engine egress services.

"""Minimal MeshFlows capability provider — Python / FastAPI."""
from __future__ import annotations

import asyncio
import os
from contextlib import asynccontextmanager
from typing import Any

import httpx
from fastapi import FastAPI
from pydantic import BaseModel, Field

SERVICE_NAME = "example.echo"
SERVICE_URL = os.getenv("SERVICE_URL", "http://localhost:8080")
REGISTRY_URL = (
    os.getenv("CAPABILITY_REGISTRY_URL")
    or os.getenv("REGISTRY_URL")
    or "http://localhost:8093"  # host publish; in-cluster: http://capability-registry:8080
).rstrip("/")
TOKEN = os.getenv("CAPABILITY_REGISTRY_TOKEN", "").strip()
LEASE_TTL_SECONDS = int(os.getenv("LEASE_TTL_SECONDS", "90"))

service_id: str | None = None
service_generation = 0
_capabilities_payload: dict[str, Any] | None = None
_heartbeat_task: asyncio.Task | None = None


def _headers() -> dict[str, str]:
    return {"Authorization": f"Bearer {TOKEN}"} if TOKEN else {}


def registration_payload() -> dict[str, Any]:
    return {
        "service_name": SERVICE_NAME,
        "service_type": "provider",
        "service_url": SERVICE_URL,
        "description": "Minimal echo capability provider.",
        "icon_emoji": "🔁",
        "capabilities": [
            {
                "name": "example.echo",
                "version": "1.0.0",
                "kind": "workflow_step",
                "description": "Echo a message back to the workflow.",
                "metadata": {
                    "public_paths": [
                        {
                            "path": "/echo",
                            "method": "POST",
                            "auth": "none",
                            "max_payload_bytes": 1048576,
                        }
                    ],
                    "designer": {
                        "display_name": "Echo message",
                        "category": "examples",
                        "step_type": "service_call",
                        "icon_emoji": "🔁",
                        "description": "Returns the input message.",
                        "form_fields": [
                            {
                                "name": "capability_name",
                                "label": "Capability name",
                                "type": "text",
                                "required": True,
                                "default": "example.echo",
                                "yaml_path": "capability_name",
                            },
                            {
                                "name": "path",
                                "label": "Path on provider",
                                "type": "text",
                                "default": "/echo",
                                "yaml_path": "path",
                            },
                            {
                                "name": "message",
                                "label": "Message",
                                "type": "text",
                                "required": True,
                                "yaml_path": "message",
                            },
                        ],
                    },
                },
            }
        ],
    }


async def register_service() -> None:
    global service_id, service_generation, _capabilities_payload
    payload = registration_payload()
    _capabilities_payload = payload
    async with httpx.AsyncClient() as client:
        r = await client.post(
            f"{REGISTRY_URL}/v1/registry/services/register",
            json=payload,
            headers=_headers(),
            timeout=5.0,
        )
        r.raise_for_status()
        body = r.json()
        service_id = body.get("service_id")
        service_generation = int(body.get("generation") or 0)


async def heartbeat_loop() -> None:
    global service_id, service_generation
    while True:
        await asyncio.sleep(max(LEASE_TTL_SECONDS / 2, 5))
        try:
            if not service_id:
                await register_service()
                continue
            async with httpx.AsyncClient() as client:
                r = await client.post(
                    f"{REGISTRY_URL}/v1/registry/services/{service_id}/heartbeat",
                    json={"generation": service_generation},
                    headers=_headers(),
                    timeout=5.0,
                )
                if r.status_code in (404, 409):
                    service_id = None
                    service_generation = 0
                    await register_service()
                    continue
                r.raise_for_status()
                service_generation = int(r.json().get("generation") or service_generation)
        except Exception:
            # Keep looping; next tick retries register/heartbeat.
            pass


@asynccontextmanager
async def lifespan(_app: FastAPI):
    global _heartbeat_task
    try:
        await register_service()
    except Exception:
        pass
    _heartbeat_task = asyncio.create_task(heartbeat_loop())
    yield
    if _heartbeat_task:
        _heartbeat_task.cancel()


app = FastAPI(title="example.echo", lifespan=lifespan)


class EchoRequest(BaseModel):
    message: str = Field(..., min_length=1)


@app.post("/echo")
def echo(body: EchoRequest) -> dict[str, str]:
    return {"echo": body.message}


@app.get("/internal/capabilities")
def capabilities() -> dict[str, Any]:
    return _capabilities_payload or {"capabilities": []}


@app.get("/healthz")
def healthz() -> dict[str, str]:
    return {"status": "ok"}

Production reference implementations live under engine/services/ (for example HTTP and Weatherstack egress).

// Minimal MeshFlows capability provider — Go.
package main

import (
    "bytes"
    "context"
    "encoding/json"
    "log"
    "net/http"
    "os"
    "strconv"
    "sync"
    "time"
)

func env(k, def string) string {
    if v := os.Getenv(k); v != "" {
        return v
    }
    return def
}

func main() {
    serviceURL := env("SERVICE_URL", "http://localhost:8080")
    registryURL := env("CAPABILITY_REGISTRY_URL", env("REGISTRY_URL", "http://localhost:8093"))
    token := os.Getenv("CAPABILITY_REGISTRY_TOKEN")
    leaseTTL, _ := strconv.Atoi(env("LEASE_TTL_SECONDS", "90"))
    if leaseTTL <= 0 {
        leaseTTL = 90
    }

    var (
        mu           sync.Mutex
        serviceID    string
        generation   int
        capabilities = registrationPayload(serviceURL)
    )

    headers := func(req *http.Request) {
        if token != "" {
            req.Header.Set("Authorization", "Bearer "+token)
        }
        req.Header.Set("Content-Type", "application/json")
    }

    register := func(ctx context.Context) error {
        body, _ := json.Marshal(capabilities)
        req, err := http.NewRequestWithContext(ctx, http.MethodPost, registryURL+"/v1/registry/services/register", bytes.NewReader(body))
        if err != nil {
            return err
        }
        headers(req)
        resp, err := http.DefaultClient.Do(req)
        if err != nil {
            return err
        }
        defer resp.Body.Close()
        if resp.StatusCode >= 300 {
            return errStatus(resp.StatusCode)
        }
        var out struct {
            ServiceID  string `json:"service_id"`
            Generation int    `json:"generation"`
        }
        if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
            return err
        }
        mu.Lock()
        serviceID, generation = out.ServiceID, out.Generation
        mu.Unlock()
        return nil
    }

    go func() {
        _ = register(context.Background())
        t := time.NewTicker(time.Duration(leaseTTL/2) * time.Second)
        defer t.Stop()
        for range t.C {
            mu.Lock()
            id, gen := serviceID, generation
            mu.Unlock()
            if id == "" {
                _ = register(context.Background())
                continue
            }
            payload, _ := json.Marshal(map[string]int{"generation": gen})
            req, err := http.NewRequest(http.MethodPost, registryURL+"/v1/registry/services/"+id+"/heartbeat", bytes.NewReader(payload))
            if err != nil {
                continue
            }
            headers(req)
            resp, err := http.DefaultClient.Do(req)
            if err != nil {
                continue
            }
            if resp.StatusCode == 404 || resp.StatusCode == 409 {
                resp.Body.Close()
                mu.Lock()
                serviceID, generation = "", 0
                mu.Unlock()
                _ = register(context.Background())
                continue
            }
            var out struct {
                Generation int `json:"generation"`
            }
            _ = json.NewDecoder(resp.Body).Decode(&out)
            resp.Body.Close()
            if out.Generation > 0 {
                mu.Lock()
                generation = out.Generation
                mu.Unlock()
            }
        }
    }()

    mux := http.NewServeMux()
    mux.HandleFunc("/echo", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodPost {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        var in struct {
            Message string `json:"message"`
        }
        if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Message == "" {
            http.Error(w, "message required", http.StatusBadRequest)
            return
        }
        writeJSON(w, map[string]string{"echo": in.Message})
    })
    mux.HandleFunc("/internal/capabilities", func(w http.ResponseWriter, r *http.Request) {
        writeJSON(w, capabilities)
    })
    mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
        writeJSON(w, map[string]string{"status": "ok"})
    })

    log.Println("example.echo listening on :8080")
    log.Fatal(http.ListenAndServe(":8080", mux))
}

func registrationPayload(serviceURL string) map[string]any {
    return map[string]any{
        "service_name": "example.echo",
        "service_type": "provider",
        "service_url":  serviceURL,
        "description":  "Minimal echo capability provider.",
        "icon_emoji":   "🔁",
        "capabilities": []any{
            map[string]any{
                "name":        "example.echo",
                "version":     "1.0.0",
                "kind":        "workflow_step",
                "description": "Echo a message back to the workflow.",
                "metadata": map[string]any{
                    "public_paths": []any{
                        map[string]any{"path": "/echo", "method": "POST", "auth": "none", "max_payload_bytes": 1048576},
                    },
                    "designer": map[string]any{
                        "display_name": "Echo message",
                        "category":     "examples",
                        "step_type":    "service_call",
                        "icon_emoji":   "🔁",
                        "form_fields": []any{
                            map[string]any{"name": "capability_name", "label": "Capability name", "type": "text", "required": true, "default": "example.echo", "yaml_path": "capability_name"},
                            map[string]any{"name": "path", "label": "Path on provider", "type": "text", "default": "/echo", "yaml_path": "path"},
                            map[string]any{"name": "message", "label": "Message", "type": "text", "required": true, "yaml_path": "message"},
                        },
                    },
                },
            },
        },
    }
}

func writeJSON(w http.ResponseWriter, v any) {
    w.Header().Set("Content-Type", "application/json")
    _ = json.NewEncoder(w).Encode(v)
}

type statusError int

func (e statusError) Error() string { return "http status " + strconv.Itoa(int(e)) }
func errStatus(code int) error      { return statusError(code) }
// Minimal MeshFlows capability provider — C# / ASP.NET Minimal APIs.
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json.Serialization;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHttpClient();

var app = builder.Build();

var serviceUrl = Environment.GetEnvironmentVariable("SERVICE_URL") ?? "http://localhost:8080";
var registryUrl = (Environment.GetEnvironmentVariable("CAPABILITY_REGISTRY_URL")
    ?? Environment.GetEnvironmentVariable("REGISTRY_URL")
    ?? "http://localhost:8093").TrimEnd('/');
var token = Environment.GetEnvironmentVariable("CAPABILITY_REGISTRY_TOKEN") ?? "";
var leaseTtl = int.TryParse(Environment.GetEnvironmentVariable("LEASE_TTL_SECONDS"), out var ttl) ? ttl : 90;

var capabilities = RegistrationPayload(serviceUrl);
string? serviceId = null;
var generation = 0;

async Task RegisterAsync(HttpClient http, CancellationToken ct)
{
    using var req = new HttpRequestMessage(HttpMethod.Post, $"{registryUrl}/v1/registry/services/register")
    {
        Content = JsonContent.Create(capabilities)
    };
    if (!string.IsNullOrEmpty(token))
        req.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);

    using var resp = await http.SendAsync(req, ct);
    resp.EnsureSuccessStatusCode();
    var body = await resp.Content.ReadFromJsonAsync<RegisterResponse>(cancellationToken: ct);
    serviceId = body?.ServiceId;
    generation = body?.Generation ?? 0;
}

_ = Task.Run(async () =>
{
    var http = new HttpClient();
    try { await RegisterAsync(http, CancellationToken.None); } catch { /* retry on heartbeat */ }

    while (true)
    {
        await Task.Delay(TimeSpan.FromSeconds(Math.Max(leaseTtl / 2.0, 5)));
        try
        {
            if (string.IsNullOrEmpty(serviceId))
            {
                await RegisterAsync(http, CancellationToken.None);
                continue;
            }

            using var req = new HttpRequestMessage(HttpMethod.Post, $"{registryUrl}/v1/registry/services/{serviceId}/heartbeat")
            {
                Content = JsonContent.Create(new { generation })
            };
            if (!string.IsNullOrEmpty(token))
                req.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);

            using var resp = await http.SendAsync(req);
            if (resp.StatusCode is System.Net.HttpStatusCode.NotFound or System.Net.HttpStatusCode.Conflict)
            {
                serviceId = null;
                generation = 0;
                await RegisterAsync(http, CancellationToken.None);
                continue;
            }
            resp.EnsureSuccessStatusCode();
            var body = await resp.Content.ReadFromJsonAsync<RegisterResponse>();
            if (body?.Generation is int g and > 0)
                generation = g;
        }
        catch
        {
            // Keep looping; next tick retries.
        }
    }
});

app.MapPost("/echo", (EchoRequest body) =>
{
    if (string.IsNullOrWhiteSpace(body.Message))
        return Results.BadRequest(new { error = "message required" });
    return Results.Ok(new { echo = body.Message });
});

app.MapGet("/internal/capabilities", () => Results.Ok(capabilities));
app.MapGet("/healthz", () => Results.Ok(new { status = "ok" }));

app.Run("http://0.0.0.0:8080");

static object RegistrationPayload(string serviceUrl) => new
{
    service_name = "example.echo",
    service_type = "provider",
    service_url = serviceUrl,
    description = "Minimal echo capability provider.",
    icon_emoji = "🔁",
    capabilities = new object[]
    {
        new
        {
            name = "example.echo",
            version = "1.0.0",
            kind = "workflow_step",
            description = "Echo a message back to the workflow.",
            metadata = new
            {
                public_paths = new object[]
                {
                    new { path = "/echo", method = "POST", auth = "none", max_payload_bytes = 1048576 }
                },
                designer = new
                {
                    display_name = "Echo message",
                    category = "examples",
                    step_type = "service_call",
                    icon_emoji = "🔁",
                    form_fields = new object[]
                    {
                        new { name = "capability_name", label = "Capability name", type = "text", required = true, @default = "example.echo", yaml_path = "capability_name" },
                        new { name = "path", label = "Path on provider", type = "text", @default = "/echo", yaml_path = "path" },
                        new { name = "message", label = "Message", type = "text", required = true, yaml_path = "message" }
                    }
                }
            }
        }
    }
};

record EchoRequest(string Message);
record RegisterResponse(
    [property: JsonPropertyName("service_id")] string? ServiceId,
    [property: JsonPropertyName("generation")] int Generation
);

Use it from a workflow

This is also how you add a C# custom process to a workflow: compile your logic into the provider, register a capability, then call it with service_call. MeshFlows does not run inline C# in SES — see C# Custom Process.

name: echo-demo
description: Calls the example.echo.csharp capability provider (or example.echo.go).

steps:
  - id: echo
    type: service_call
    capability_name: example.echo.csharp
    capability_version: "^1.0"
    path: /echo
    message: "hello from MeshFlows"

Shipped workflows: flows/workflows/example_echo_csharp_demo.yaml and flows/workflows/example_echo_go_demo.yaml.

Attach triggers as usual (HTTP, schedule, AMQP) so the same workflow can start from multiple channels — see Workflow YAML.


Checklist before you ship

  1. Registration + heartbeat work against your cluster registry (CAPABILITY_REGISTRY_TOKEN if required).
  2. service_url / SERVICE_URL is reachable from Orchestrator (service DNS, not localhost, in Kubernetes).
  3. Capability name / version / kind and metadata.designer match the contract. For steps, kind: workflow_step or kind: egress both work with service_call (lookup is by name).
  4. Workflow service_call uses the same capability_name and path.
  5. If the provider needs credentials: register kind: workflow_connection, add flows/connections/<type>.yaml with $secret:…, and pass connection: on the step — see Integration end-to-end.
  6. Prefer a trigger under flows/triggers/ with target_workflow (demo pattern).
  7. For Engine-style deploy (images, kustomize, CI), follow Deploy integration and set capability_provider: true in engine/deploy/engine-images-manifest.yaml.