Tutorial

Agent Pipelines

Build a four-stage pipeline — filter, enrich, score, deliver — that turns noisy webhooks into high-signal events for an agent.

A raw webhook stream is too noisy to hand straight to an agent. Kite's manifest models a four-stage pipeline you can shape without writing any delivery code yourself.

text

events ──▶ filter ──▶ enrich ──▶ score ──▶ sink
 (server)  (drop/keep) (augment)  (importance)  (deliver)

Each stage is optional. Start small, add stages as the stream gets louder.

By the end of this tutorial you'll have a kite.json that watches GitHub pushes, drops bot pushes, augments auth-path changes with a blame summary, and only pages an agent when something lands in a sensitive directory.

// 011. Filter — drop obvious noise first

filters runs before anything expensive. Use it to throw away pings, bot activity, and event types the agent doesn't care about.

json

{
  "name": "code-sentinel",
  "subscriptions": [{ "source": "github" }],
  "filters": {
    "drop": [
      { "source": "github", "type": "com.github.ping" },
      { "actor": "dependabot\\[bot\\]" },
      { "actor": "renovate\\[bot\\]" }
    ],
    "keep_only": [
      { "source": "github", "type": "com.github.push" },
      { "source": "github", "type": "com.github.pull_request.*" }
    ]
  },
  "sink": { "type": "stdout", "json": true }
}

Run this and watch stdout. If you still see events you don't want, add them to drop.

// 022. Enrich — attach context the event doesn't carry

The CloudEvent payload from GitHub knows what changed but not *why it matters*. The enrichment stage lets you run a shell script per matched event, read the event JSON on stdin, and emit metadata for later scoring.

json

{
  "enrichment": [
    {
      "match": { "source": "github", "type": "com.github.push" },
      "run": "./enrich/blame-summary.sh",
      "timeout": "30s"
    }
  ]
}

A minimal enrich/blame-summary.sh:

bash

#!/usr/bin/env bash
set -euo pipefail
event_json=$(cat)
sha=$(jq -r '.data.after' <<< "$event_json")
changed_paths=$(jq -r '.data.commits[].modified[]?' <<< "$event_json" | sort -u)
jq -n --arg summary "$(echo "$changed_paths" | head -c 2000)" \
      --argjson paths "$(printf '%s\n' "$changed_paths" | jq -R . | jq -s .)" \
      '{ blame_summary: $summary, changed_files: $paths }'

Output JSON is stored as enrichment metadata. Later scoring can use fields such as changed_files; sinks still receive the original event shape.

// 033. Score — mark what the agent should actually react to

scoring assigns an importance level to each event. The paths clause checks file globs inside the GitHub payload so you can single out sensitive directories.

json

{
  "scoring": {
    "default_importance": "normal",
    "rules": [
      {
        "match": { "source": "github", "type": "com.github.push" },
        "paths": ["src/auth/**", "migrations/**", ".github/workflows/**"],
        "importance": "critical",
        "reason": "security-sensitive path"
      },
      {
        "match": { "source": "github", "type": "com.github.pull_request.opened" },
        "importance": "high",
        "reason": "new PR"
      }
    ]
  }
}

// 044. Deliver — gate the sink on importance

Finally, hand the survivors to the agent. Use the sink's own importance threshold so normal-noise events don't reach it at all.

json

{
  "sink": {
    "type": "exec",
    "command": "./agent/dispatch.sh",
    "importance": "high"
  }
}

Replace exec with paperclip to fan straight into a Paperclip agent, or mcp_server to expose the stream to a local MCP-aware agent.

// 05Putting it together

The full pipeline manifest:

json

{
  "name": "code-sentinel",
  "subscriptions": [{ "source": "github" }],
  "filters": {
    "drop": [
      { "source": "github", "type": "com.github.ping" },
      { "actor": "dependabot\\[bot\\]" },
      { "actor": "renovate\\[bot\\]" }
    ],
    "keep_only": [
      { "source": "github", "type": "com.github.push" },
      { "source": "github", "type": "com.github.pull_request.*" }
    ]
  },
  "enrichment": [
    {
      "match": { "source": "github", "type": "com.github.push" },
      "run": "./enrich/blame-summary.sh",
      "timeout": "30s"
    }
  ],
  "scoring": {
    "default_importance": "normal",
    "rules": [
      {
        "match": { "source": "github", "type": "com.github.push" },
        "paths": ["src/auth/**", "migrations/**"],
        "importance": "critical",
        "reason": "security-sensitive path"
      }
    ]
  },
  "sink": {
    "type": "exec",
    "command": "./agent/dispatch.sh",
    "importance": "high"
  }
}

Run it:

bash

kite run --manifest kite.json

// 06What the agent sees

A trickle of high/critical events, not a firehose.
Scoring has already considered enrichment fields such as changed_files.
Every delivered event carries the reason label from scoring.

From here, dispatch.sh is a normal shell handler. It reads the event JSON on stdin and kicks off whatever your agent does — open a PR, message a Slack channel, file a Linear issue, page oncall.

// 07See also

Filtering Events — deeper treatment of each filter layer
Manifest Schema — every field with type and default
Paperclip Integration — swap the exec sink for a paperclip sink