Introducing Kite Federation: Route Events Across Any Kite Instance

As Kite has grown, we've seen a consistent pattern: teams start with a single Kite instance, then reach the point where they need events to flow across multiple environments — local dev, staging, production, and across Paperclip agent clusters running on separate machines.

Until now, that required custom glue: HTTP forwarding scripts, message queues, or just duplicating webhook configurations across every environment. Today, we're shipping Kite Federation to solve this cleanly.

The Multi-Instance Problem

Suppose you have two Kite servers: one self-hosted on your internal network, one cloud-managed. A Stripe payment_intent.succeeded event arrives at your cloud instance. Your fulfillment agent runs on the self-hosted instance. Getting that event across the boundary today requires you to build and maintain your own bridge.

Federation makes this a first-class feature.

Server-Side Hub Federation

Kite Federation uses a server-side hub model — not CLI-to-CLI P2P.

Here's why that matters: P2P federation puts delivery responsibility on ephemeral CLI processes. If a CLI disconnects, events are lost unless the client implements its own DLQ. With server-side federation, your Kite server holds a Postgres-backed outbox queue. Events are durably enqueued the moment they're ingested — delivery happens asynchronously, survives restarts, and retries automatically with exponential backoff.

┌──────────────┐                         ┌──────────────┐
│ Kite Server  │   federation_peers      │ Kite Server  │
│ (Instance A) │◄═══════════════════════►│ (Instance B) │
│              │  mutual API keys        │              │
│  events tbl  │  server-to-server POST  │  events tbl  │
└──────┬───────┘                         └──────┬───────┘
       │ WebSocket                              │ WebSocket
       ▼                                        ▼
   CLI clients                              CLI clients
   (A's agents)                             (B's agents)

Delivery flow:

1. Webhook arrives at Instance A → ingested as CloudEvent → stored → broadcast to local listeners 2. Server checks federation_peers for subscribed peers 3. Matching peers get a durable federation_outbox entry 4. Background worker POSTs the CloudEvent to the peer's hook endpoint 5. Peer ingests normally (HMAC verification, dedup, store, broadcast) 6. Delivery status tracked in federation_deliveries for retry and monitoring

CloudEvents Provenance Chain

Every federated event carries three new extension attributes:

| Attribute | Type | Purpose | |-----------|------|---------| | kitefedorigin | string | Instance ID of the originating server | | kitefedchain | string (JSON array) | Ordered list of instance IDs traversed | | kitefedhops | integer | Number of federation hops |

These extensions serve double duty: they give you full event provenance for debugging, and they're the foundation of loop detection. Before enqueueing a federated event, the server checks if the destination peer's instance ID already appears in kitefedchain. If yes, the event is dropped. A configurable max-hops limit (default: 3) provides an additional backstop.

Agent-to-Agent Messaging

Federation isn't just for webhook forwarding. It unlocks a new event type: `com.kite.agent.message`.

Agents on different Paperclip instances can now exchange structured messages via the Kite event mesh:

json

{
  "specversion": "1.0",
  "type": "com.kite.agent.message",
  "source": "kite://instance-1/agents/agent-a-id",
  "id": "018f...",
  "data": {
    "from_agent_id": "uuid",
    "to_agent_id": "uuid",
    "message_type": "task_handoff",
    "payload": { "task": "process_payment", "context": { ... } },
    "correlation_id": "uuid"
  }
}

Combined with the existing PaperclipSink, this gives you cross-instance task handoffs, status updates, and data sharing — all delivered via the same reliable federation layer.

Security Model

Federation is designed to be zero-trust by default:

•TLS only — non-HTTPS peer URLs are rejected at registration time
•Mutual authentication — both peers verify each other on every request using bcrypt-hashed tokens
•Team-scoped isolation — federated events land in a specific team, never crossing team boundaries
•Per-peer rate limits — configurable (default: 100 events/sec) to prevent cascade amplification
•Replay protection — original event_id dedup prevents forwarded events from being ingested twice

The server holds federation credentials centrally (not scattered across CLI processes), which significantly reduces the credential exposure surface.

CLI P2P for Local Dev

For local development and testing, CLI-level P2P federation is available as an escape hatch:

bash

kite stream \
  --federation-target https://other-instance.example.com/hooks/<team>/kite \
  --federation-token <outbound-token> \
  --instance-id my-local-dev

This is lightweight (no server config, no DB migration) but provides best-effort delivery — suitable for dev/test, not production.

Getting Started

Register a peer via the API. On Instance A, create a peer record for Instance B — the server generates a token pair and returns both in plaintext once:

bash

curl -X POST "https://a.kite.example.com/api/federation/peers?team_id=<team-id>" \
  -H "Authorization: Bearer <api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "peer_instance_id": "kite-instance-b",
    "peer_hook_url": "https://b.kite.example.com/hooks/<b-team>/kite",
    "scopes": ["stripe.*", "github.*"]
  }'

Exchange the returned outbound_token and inbound_token with the Instance B admin so both sides can validate each other. Then repeat on Instance B for the reverse direction.

Events matching the configured scopes now flow automatically between instances. See the [Federation docs](/docs/concepts/federation) for the full setup guide.

What's Next

Federation ships as part of the core Kite server. Peer management UI (dashboard pages for peer health, delivery metrics, and dead-letter inspection) is coming in the next release.

We're also exploring federation mesh topologies — automatic peer discovery for teams running Kite at scale across many environments. If that's relevant to your use case, [let us know](mailto:[email protected]).