Add a Capability Provider¶
Extend MeshFlows without changing the Orchestrator. Register a small HTTP service with the capability registry; workflows call it through service_call steps. The Dashboard designer picks up metadata.designer for forms and labels.
This page is the starter kit: a minimal provider in Python, Go, and C#, plus the matching workflow YAML. For the full happy path (connection type, secrets, trigger, deploy checklist), see Integration end-to-end. For field-level rules, see the Capability provider contract. For Kubernetes / CI wiring, see Deploy integration.
Canonical Engine providers (Weatherstack, HTTP egress, …) are written in Python/FastAPI — use that stack in production MeshFlows images. Go and C# examples show that any language that speaks HTTP can implement the same contract.
Shipped demos (prefer these over pasting the snippets below when learning):
| Language | Service | Capability | Workflow / HTTP |
|---|---|---|---|
| C# | engine/services/example-echo-csharp/ |
example.echo.csharp |
example_echo_csharp_demo → POST /v1/api/examples/echo-csharp |
| Go | engine/services/example-echo-go/ |
example.echo.go |
example_echo_go_demo → POST /v1/api/examples/echo-go |
The echo sample has no credentials. Real egress providers also register a workflow_connection capability and ship flows/connections/<type>.yaml — see Integration end-to-end and Egress Weatherstack.
What you build¶
| Piece | Purpose |
|---|---|
POST /v1/registry/services/register |
Announce service_name, service_url, and capabilities[] |
| Heartbeat | POST /v1/registry/services/{service_id}/heartbeat about every LEASE_TTL/2 seconds; re-register on 404/409 |
| Capability HTTP API | The path workflows call (here: POST /echo) |
GET /internal/capabilities |
Local copy of the registration payload (ops / debugging) |
| Workflow step | type: service_call + capability_name + path |
Optional but recommended: Authorization: Bearer <CAPABILITY_REGISTRY_TOKEN> when the registry requires a token.
Shared registration payload¶
All three language samples register the same capability:
{
"service_name": "example.echo",
"service_type": "provider",
"service_url": "http://example-echo:8080",
"description": "Minimal echo capability provider.",
"icon_emoji": "🔁",
"capabilities": [
{
"name": "example.echo",
"version": "1.0.0",
"kind": "workflow_step",
"description": "Echo a message back to the workflow.",
"metadata": {
"public_paths": [
{ "path": "/echo", "method": "POST", "auth": "none", "max_payload_bytes": 1048576 }
],
"designer": {
"display_name": "Echo message",
"category": "examples",
"step_type": "service_call",
"icon_emoji": "🔁",
"description": "Returns the input message — useful as a capability hello-world.",
"form_fields": [
{
"name": "capability_name",
"label": "Capability name",
"type": "text",
"required": true,
"default": "example.echo",
"yaml_path": "capability_name"
},
{
"name": "path",
"label": "Path on provider",
"type": "text",
"default": "/echo",
"yaml_path": "path"
},
{
"name": "message",
"label": "Message",
"type": "text",
"required": true,
"yaml_path": "message",
"placeholder": "hello from MeshFlows"
}
]
}
}
}
]
}
Set service_url to the URL other cluster services use to reach your provider (Kubernetes service DNS in-cluster).
Environment¶
| Variable | Example | Notes |
|---|---|---|
SERVICE_URL |
http://example-echo:8080 |
Advertised base URL in the registry (in-cluster DNS, not localhost) |
REGISTRY_URL or CAPABILITY_REGISTRY_URL |
http://capability-registry:8080 |
In-cluster / Compose service DNS (container listens on 8080) |
CAPABILITY_REGISTRY_TOKEN |
(secret) | Optional Bearer token |
LEASE_TTL_SECONDS |
90 |
Match registry lease; heartbeat ≈ half |
Compose publishes the registry on the host as localhost:8093 → container 8080. Use http://localhost:8093 only from the host (curl/browser). Providers and Orchestrator inside the Compose network must use http://capability-registry:8080.
Implementation examples¶
Choose a language. All three samples implement the same example.echo contract (register, heartbeat, POST /echo).
Canonical Engine providers ship as Python / FastAPI; Go and C# show that any HTTP service can follow the contract.
Matches the style used by Engine egress services.
"""Minimal MeshFlows capability provider — Python / FastAPI."""
from __future__ import annotations
import asyncio
import os
from contextlib import asynccontextmanager
from typing import Any
import httpx
from fastapi import FastAPI
from pydantic import BaseModel, Field
SERVICE_NAME = "example.echo"
SERVICE_URL = os.getenv("SERVICE_URL", "http://localhost:8080")
REGISTRY_URL = (
os.getenv("CAPABILITY_REGISTRY_URL")
or os.getenv("REGISTRY_URL")
or "http://localhost:8093" # host publish; in-cluster: http://capability-registry:8080
).rstrip("/")
TOKEN = os.getenv("CAPABILITY_REGISTRY_TOKEN", "").strip()
LEASE_TTL_SECONDS = int(os.getenv("LEASE_TTL_SECONDS", "90"))
service_id: str | None = None
service_generation = 0
_capabilities_payload: dict[str, Any] | None = None
_heartbeat_task: asyncio.Task | None = None
def _headers() -> dict[str, str]:
return {"Authorization": f"Bearer {TOKEN}"} if TOKEN else {}
def registration_payload() -> dict[str, Any]:
return {
"service_name": SERVICE_NAME,
"service_type": "provider",
"service_url": SERVICE_URL,
"description": "Minimal echo capability provider.",
"icon_emoji": "🔁",
"capabilities": [
{
"name": "example.echo",
"version": "1.0.0",
"kind": "workflow_step",
"description": "Echo a message back to the workflow.",
"metadata": {
"public_paths": [
{
"path": "/echo",
"method": "POST",
"auth": "none",
"max_payload_bytes": 1048576,
}
],
"designer": {
"display_name": "Echo message",
"category": "examples",
"step_type": "service_call",
"icon_emoji": "🔁",
"description": "Returns the input message.",
"form_fields": [
{
"name": "capability_name",
"label": "Capability name",
"type": "text",
"required": True,
"default": "example.echo",
"yaml_path": "capability_name",
},
{
"name": "path",
"label": "Path on provider",
"type": "text",
"default": "/echo",
"yaml_path": "path",
},
{
"name": "message",
"label": "Message",
"type": "text",
"required": True,
"yaml_path": "message",
},
],
},
},
}
],
}
async def register_service() -> None:
global service_id, service_generation, _capabilities_payload
payload = registration_payload()
_capabilities_payload = payload
async with httpx.AsyncClient() as client:
r = await client.post(
f"{REGISTRY_URL}/v1/registry/services/register",
json=payload,
headers=_headers(),
timeout=5.0,
)
r.raise_for_status()
body = r.json()
service_id = body.get("service_id")
service_generation = int(body.get("generation") or 0)
async def heartbeat_loop() -> None:
global service_id, service_generation
while True:
await asyncio.sleep(max(LEASE_TTL_SECONDS / 2, 5))
try:
if not service_id:
await register_service()
continue
async with httpx.AsyncClient() as client:
r = await client.post(
f"{REGISTRY_URL}/v1/registry/services/{service_id}/heartbeat",
json={"generation": service_generation},
headers=_headers(),
timeout=5.0,
)
if r.status_code in (404, 409):
service_id = None
service_generation = 0
await register_service()
continue
r.raise_for_status()
service_generation = int(r.json().get("generation") or service_generation)
except Exception:
# Keep looping; next tick retries register/heartbeat.
pass
@asynccontextmanager
async def lifespan(_app: FastAPI):
global _heartbeat_task
try:
await register_service()
except Exception:
pass
_heartbeat_task = asyncio.create_task(heartbeat_loop())
yield
if _heartbeat_task:
_heartbeat_task.cancel()
app = FastAPI(title="example.echo", lifespan=lifespan)
class EchoRequest(BaseModel):
message: str = Field(..., min_length=1)
@app.post("/echo")
def echo(body: EchoRequest) -> dict[str, str]:
return {"echo": body.message}
@app.get("/internal/capabilities")
def capabilities() -> dict[str, Any]:
return _capabilities_payload or {"capabilities": []}
@app.get("/healthz")
def healthz() -> dict[str, str]:
return {"status": "ok"}
Production reference implementations live under engine/services/ (for example HTTP and Weatherstack egress).
// Minimal MeshFlows capability provider — Go.
package main
import (
"bytes"
"context"
"encoding/json"
"log"
"net/http"
"os"
"strconv"
"sync"
"time"
)
func env(k, def string) string {
if v := os.Getenv(k); v != "" {
return v
}
return def
}
func main() {
serviceURL := env("SERVICE_URL", "http://localhost:8080")
registryURL := env("CAPABILITY_REGISTRY_URL", env("REGISTRY_URL", "http://localhost:8093"))
token := os.Getenv("CAPABILITY_REGISTRY_TOKEN")
leaseTTL, _ := strconv.Atoi(env("LEASE_TTL_SECONDS", "90"))
if leaseTTL <= 0 {
leaseTTL = 90
}
var (
mu sync.Mutex
serviceID string
generation int
capabilities = registrationPayload(serviceURL)
)
headers := func(req *http.Request) {
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
req.Header.Set("Content-Type", "application/json")
}
register := func(ctx context.Context) error {
body, _ := json.Marshal(capabilities)
req, err := http.NewRequestWithContext(ctx, http.MethodPost, registryURL+"/v1/registry/services/register", bytes.NewReader(body))
if err != nil {
return err
}
headers(req)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return errStatus(resp.StatusCode)
}
var out struct {
ServiceID string `json:"service_id"`
Generation int `json:"generation"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return err
}
mu.Lock()
serviceID, generation = out.ServiceID, out.Generation
mu.Unlock()
return nil
}
go func() {
_ = register(context.Background())
t := time.NewTicker(time.Duration(leaseTTL/2) * time.Second)
defer t.Stop()
for range t.C {
mu.Lock()
id, gen := serviceID, generation
mu.Unlock()
if id == "" {
_ = register(context.Background())
continue
}
payload, _ := json.Marshal(map[string]int{"generation": gen})
req, err := http.NewRequest(http.MethodPost, registryURL+"/v1/registry/services/"+id+"/heartbeat", bytes.NewReader(payload))
if err != nil {
continue
}
headers(req)
resp, err := http.DefaultClient.Do(req)
if err != nil {
continue
}
if resp.StatusCode == 404 || resp.StatusCode == 409 {
resp.Body.Close()
mu.Lock()
serviceID, generation = "", 0
mu.Unlock()
_ = register(context.Background())
continue
}
var out struct {
Generation int `json:"generation"`
}
_ = json.NewDecoder(resp.Body).Decode(&out)
resp.Body.Close()
if out.Generation > 0 {
mu.Lock()
generation = out.Generation
mu.Unlock()
}
}
}()
mux := http.NewServeMux()
mux.HandleFunc("/echo", func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
return
}
var in struct {
Message string `json:"message"`
}
if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Message == "" {
http.Error(w, "message required", http.StatusBadRequest)
return
}
writeJSON(w, map[string]string{"echo": in.Message})
})
mux.HandleFunc("/internal/capabilities", func(w http.ResponseWriter, r *http.Request) {
writeJSON(w, capabilities)
})
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
writeJSON(w, map[string]string{"status": "ok"})
})
log.Println("example.echo listening on :8080")
log.Fatal(http.ListenAndServe(":8080", mux))
}
func registrationPayload(serviceURL string) map[string]any {
return map[string]any{
"service_name": "example.echo",
"service_type": "provider",
"service_url": serviceURL,
"description": "Minimal echo capability provider.",
"icon_emoji": "🔁",
"capabilities": []any{
map[string]any{
"name": "example.echo",
"version": "1.0.0",
"kind": "workflow_step",
"description": "Echo a message back to the workflow.",
"metadata": map[string]any{
"public_paths": []any{
map[string]any{"path": "/echo", "method": "POST", "auth": "none", "max_payload_bytes": 1048576},
},
"designer": map[string]any{
"display_name": "Echo message",
"category": "examples",
"step_type": "service_call",
"icon_emoji": "🔁",
"form_fields": []any{
map[string]any{"name": "capability_name", "label": "Capability name", "type": "text", "required": true, "default": "example.echo", "yaml_path": "capability_name"},
map[string]any{"name": "path", "label": "Path on provider", "type": "text", "default": "/echo", "yaml_path": "path"},
map[string]any{"name": "message", "label": "Message", "type": "text", "required": true, "yaml_path": "message"},
},
},
},
},
},
}
}
func writeJSON(w http.ResponseWriter, v any) {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(v)
}
type statusError int
func (e statusError) Error() string { return "http status " + strconv.Itoa(int(e)) }
func errStatus(code int) error { return statusError(code) }
// Minimal MeshFlows capability provider — C# / ASP.NET Minimal APIs.
using System.Net.Http.Headers;
using System.Net.Http.Json;
using System.Text.Json.Serialization;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHttpClient();
var app = builder.Build();
var serviceUrl = Environment.GetEnvironmentVariable("SERVICE_URL") ?? "http://localhost:8080";
var registryUrl = (Environment.GetEnvironmentVariable("CAPABILITY_REGISTRY_URL")
?? Environment.GetEnvironmentVariable("REGISTRY_URL")
?? "http://localhost:8093").TrimEnd('/');
var token = Environment.GetEnvironmentVariable("CAPABILITY_REGISTRY_TOKEN") ?? "";
var leaseTtl = int.TryParse(Environment.GetEnvironmentVariable("LEASE_TTL_SECONDS"), out var ttl) ? ttl : 90;
var capabilities = RegistrationPayload(serviceUrl);
string? serviceId = null;
var generation = 0;
async Task RegisterAsync(HttpClient http, CancellationToken ct)
{
using var req = new HttpRequestMessage(HttpMethod.Post, $"{registryUrl}/v1/registry/services/register")
{
Content = JsonContent.Create(capabilities)
};
if (!string.IsNullOrEmpty(token))
req.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
using var resp = await http.SendAsync(req, ct);
resp.EnsureSuccessStatusCode();
var body = await resp.Content.ReadFromJsonAsync<RegisterResponse>(cancellationToken: ct);
serviceId = body?.ServiceId;
generation = body?.Generation ?? 0;
}
_ = Task.Run(async () =>
{
var http = new HttpClient();
try { await RegisterAsync(http, CancellationToken.None); } catch { /* retry on heartbeat */ }
while (true)
{
await Task.Delay(TimeSpan.FromSeconds(Math.Max(leaseTtl / 2.0, 5)));
try
{
if (string.IsNullOrEmpty(serviceId))
{
await RegisterAsync(http, CancellationToken.None);
continue;
}
using var req = new HttpRequestMessage(HttpMethod.Post, $"{registryUrl}/v1/registry/services/{serviceId}/heartbeat")
{
Content = JsonContent.Create(new { generation })
};
if (!string.IsNullOrEmpty(token))
req.Headers.Authorization = new AuthenticationHeaderValue("Bearer", token);
using var resp = await http.SendAsync(req);
if (resp.StatusCode is System.Net.HttpStatusCode.NotFound or System.Net.HttpStatusCode.Conflict)
{
serviceId = null;
generation = 0;
await RegisterAsync(http, CancellationToken.None);
continue;
}
resp.EnsureSuccessStatusCode();
var body = await resp.Content.ReadFromJsonAsync<RegisterResponse>();
if (body?.Generation is int g and > 0)
generation = g;
}
catch
{
// Keep looping; next tick retries.
}
}
});
app.MapPost("/echo", (EchoRequest body) =>
{
if (string.IsNullOrWhiteSpace(body.Message))
return Results.BadRequest(new { error = "message required" });
return Results.Ok(new { echo = body.Message });
});
app.MapGet("/internal/capabilities", () => Results.Ok(capabilities));
app.MapGet("/healthz", () => Results.Ok(new { status = "ok" }));
app.Run("http://0.0.0.0:8080");
static object RegistrationPayload(string serviceUrl) => new
{
service_name = "example.echo",
service_type = "provider",
service_url = serviceUrl,
description = "Minimal echo capability provider.",
icon_emoji = "🔁",
capabilities = new object[]
{
new
{
name = "example.echo",
version = "1.0.0",
kind = "workflow_step",
description = "Echo a message back to the workflow.",
metadata = new
{
public_paths = new object[]
{
new { path = "/echo", method = "POST", auth = "none", max_payload_bytes = 1048576 }
},
designer = new
{
display_name = "Echo message",
category = "examples",
step_type = "service_call",
icon_emoji = "🔁",
form_fields = new object[]
{
new { name = "capability_name", label = "Capability name", type = "text", required = true, @default = "example.echo", yaml_path = "capability_name" },
new { name = "path", label = "Path on provider", type = "text", @default = "/echo", yaml_path = "path" },
new { name = "message", label = "Message", type = "text", required = true, yaml_path = "message" }
}
}
}
}
}
};
record EchoRequest(string Message);
record RegisterResponse(
[property: JsonPropertyName("service_id")] string? ServiceId,
[property: JsonPropertyName("generation")] int Generation
);
Use it from a workflow¶
This is also how you add a C# custom process to a workflow: compile your logic into the
provider, register a capability, then call it with service_call. MeshFlows does not run
inline C# in SES — see C# Custom Process.
name: echo-demo
description: Calls the example.echo.csharp capability provider (or example.echo.go).
steps:
- id: echo
type: service_call
capability_name: example.echo.csharp
capability_version: "^1.0"
path: /echo
message: "hello from MeshFlows"
Shipped workflows: flows/workflows/example_echo_csharp_demo.yaml and
flows/workflows/example_echo_go_demo.yaml.
Attach triggers as usual (HTTP, schedule, AMQP) so the same workflow can start from multiple channels — see Workflow YAML.
Checklist before you ship¶
- Registration + heartbeat work against your cluster registry (
CAPABILITY_REGISTRY_TOKENif required). service_url/SERVICE_URLis reachable from Orchestrator (service DNS, notlocalhost, in Kubernetes).- Capability
name/version/kindandmetadata.designermatch the contract. For steps,kind: workflow_steporkind: egressboth work withservice_call(lookup is by name). - Workflow
service_calluses the samecapability_nameandpath. - If the provider needs credentials: register
kind: workflow_connection, addflows/connections/<type>.yamlwith$secret:…, and passconnection:on the step — see Integration end-to-end. - Prefer a trigger under
flows/triggers/withtarget_workflow(demo pattern). - For Engine-style deploy (images, kustomize, CI), follow Deploy integration and set
capability_provider: trueinengine/deploy/engine-images-manifest.yaml.
Related¶
- Integration end-to-end — provider + connection + workflow + trigger
- C# Custom Process in a Workflow — workflow-author view
- Capability provider contract
- Deploy integration
- Egress Weatherstack — real Python provider
- Wiki home