Outgoing HTTP notifications. An administrator registers an endpoint — a URL, a signing secret the CMS mints, and three filters — and the CMS POSTs a signed envelope to it whenever a matching entry is created, updated, published, unpublished, deleted, restored or purged. Every attempt is logged, retried on a fixed ladder, and visible in the admin.
The envelope carries references, not content. A receiver reads the record back through the public API with its own token, so the read passes through the visibility rules and audience entitlements an editor set on the entry. Inlining field values here would route around all of them at once.
Installing it changes nothing
With no endpoint registered, the fan-out subscriber finds no subscriptions and returns; the worker claims nothing. Nothing is sent, and nothing costs more than it did.
Install
npm install @apograph/webhooks-server @apograph/webhooks-adminBoth are installed in every app create-apograph-app generates, and both depend
on @apograph/webhooks-domain — the framework-free kernel a receiver can also
import to verify a signature with the same code that produced it.
Register
// apps/server/src/plugins.ts
import { WebhooksPlugin } from '@apograph/webhooks-server';
WebhooksPlugin(config.plugins.webhooks);// apps/admin/src/plugins.ts
import { WebhooksPlugin } from '@apograph/webhooks-admin';
WebhooksPlugin();The server half needs DatabasePlugin (the outbox it subscribes to) and
IdentityPlugin (the permissions its routes are gated on) ahead of it, and
belongs after ContentPlugin, whose entry events are the only ones it delivers.
It binds no port anything else uses, so the reference host registers it last
among the content-adjacent plugins, just before the copilot. The admin half
contributes to the global sidebar’s directory group beside API tokens; its
position in the list does not matter.
It owns three tables — webhook_endpoints, webhook_endpoint_workspaces and
webhook_deliveries — and ships their migrations under
__drizzle_migrations_webhooks.
Nothing is sent from the outbox subscriber
This is the one structural rule, and it is an ADR rather than an implementation detail (ADR-0016).
The outbox dispatcher calls its subscribers inside the transaction that claims a batch of events. A subscriber that made an HTTP request there would hold a database transaction and a pool client for a stranger’s response time — one slow receiver stalling the whole drain — and a thrown error would count against the outbox row’s attempt budget, eventually parking an event the activity log and the alarms evaluator were also waiting for.
So the subscriber does one SELECT over the endpoints and one batched INSERT
of delivery rows, and nothing else. A worker on a plain interval claims
rows with FOR UPDATE SKIP LOCKED, commits the claim, and only then touches
the network with nothing open. Several API processes running the worker take
disjoint rows, so scaling out simply sends faster. Deliveries within one batch
go out serially, on purpose: twenty parallel requests would be a burst aimed at
whichever receivers happen to be subscribed.
Outbox delivery is at-least-once, so the subscriber may see one event twice.
unique(endpoint_id, event_id) on the delivery row makes the repeat a no-op
rather than a second POST.
Configuration
The plugin takes an options object under the webhooks key. The endpoints
themselves are not configuration — they are operational data that changes
without a redeploy, and they live in the database.
| Option | Default | What it bounds |
|---|---|---|
deliveryIntervalMs | 2,000 | How often the worker looks for claimable rows. 0 queues but never sends from this process |
batchSize | 20 | Deliveries one tick claims — the burst one process can aim at receivers |
timeoutMs | 10,000 | Per-request timeout |
maxAttempts | 6 | Attempts before a delivery is given up on |
retentionDays | 30 | How long a finished delivery stays in the log. 0 keeps them forever |
autoDisableAfter | 20 | Consecutive dead deliveries that switch an endpoint off |
allowInsecureUrls | false | Permit http:// destinations |
allowPrivateNetworks | false | Permit loopback, link-local and RFC 1918 destinations |
claimTimeoutMs | 30,000 | How long a row may sit in delivering before a crashed worker’s claim is reclaimed. Must exceed timeoutMs |
responseSnippetBytes | 2,048 | Bytes of the receiver’s response kept in the log |
A batchSize of zero, a claimTimeoutMs at or below timeoutMs, or a negative
anything is refused at construction: the first is a worker that looks exactly
like one with nothing to do, and the second reclaims deliveries out from under
live requests and sends every slow one twice.
The CMS’s own host reads five of these from the environment
(apps/server/config/webhooks.ts); the rest are left to the defaults, which is
where the reasoning behind them is written down.
| Variable | Default | Sets |
|---|---|---|
WEBHOOKS_DELIVERY_INTERVAL | 2000 | deliveryIntervalMs |
WEBHOOKS_TIMEOUT | 10000 | timeoutMs |
WEBHOOKS_RETENTION_DAYS | 30 | retentionDays |
WEBHOOKS_ALLOW_INSECURE_URLS | false | allowInsecureUrls |
WEBHOOKS_ALLOW_PRIVATE_NETWORKS | false | allowPrivateNetworks |
An app generated before 0.5.2 writes these and reads none of them
From 0.5.2 a generated app ships apps/server/config/webhooks.ts, a
webhooks entry in apograph.config.ts, and
WebhooksPlugin(config.plugins.webhooks) — the five variables work as
documented.
Before that, create-apograph-app put them in the app’s .env and generated
WebhooksPlugin() with no argument, so editing one changed nothing and nothing
said so. If your app was scaffolded by 0.5.1 or earlier, add the module and pass
it; the upgrade guide has the steps. The two allow*
flags are the ones this actually bites — see
configuration.
The event catalogue
Eight kinds, and the catalogue is data, not a switch: the admin’s picker,
the DTO validation and GET /api/webhook-events all read one array, so a
deployment running a newer server offers the kinds that server actually knows.
| Kind | Group | Filtered by content type | Carries a workspace |
|---|---|---|---|
entry.created | content | yes | yes |
entry.updated | content | yes | yes |
entry.published | content | yes | yes |
entry.unpublished | content | yes | yes |
entry.deleted | content | yes | yes |
entry.restored | content | yes | yes |
entry.purged | content | yes | yes |
ping | system | no | no |
ping is the synthetic event the Send test button produces. It is
addressed to one endpoint by hand, sent synchronously, never queued and never
fanned out — and it does not count against the endpoint’s failure budget.
Media, transfer and account events exist on the outbox and are deliberately not here: media events do not yet carry their workspace on the payload, and account or workspace events are administrative audit rather than content changes, with a different audience and a different sensitivity.
Subscriptions are three sets, and empty means everything
An endpoint names the workspaces, event kinds and content types it takes, and an event is delivered when it is in all three. There is no expression language — a second query grammar would be a second thing to document and a second place for a rule to mean something the UI does not.
An empty set means everything, including what does not exist yet. “All kinds” survives a new kind being added to the catalogue; “all types” survives a new content type. That is the opposite reading from an API token’s workspace bucket, which forbids an empty set — but a token’s set is the bounds of its authority, where “all” would be a hole, while this one is a filter, where “all” is an ordinary answer.
The one deliberate asymmetry: all workspaces is a stored flag
(allWorkspaces), not an inference from an empty list, and it defaults to
false. Reaching across every tenant should be something someone chose. An
entry with no workspace reaches only endpoints that take them all.
The envelope
Every delivery is a POST with Content-Type: application/json, a
User-Agent of Apograph-Webhooks/1, and six headers of its own:
| Header | Carries |
|---|---|
X-Apograph-Event | The event kind, so a receiver can route without parsing the body |
X-Apograph-Delivery | This delivery’s id. A redelivery gets a new one |
X-Apograph-Event-Id | The originating outbox event’s id. Stable across retries and redeliveries — deduplicate on this |
X-Apograph-Workspace | The owning workspace. Omitted when the event has none |
X-Apograph-Attempt | Which attempt this is, 1-based |
X-Apograph-Signature | t=<unix seconds>,v1=<hex hmac> |
The body is the same for every kind:
{
"id": "0f1c9a2e-…",
"event": "entry.published",
"eventId": "9c41d3b8-…",
"occurredAt": "2026-09-06T10:12:04.881Z",
"workspaceId": "b71e4f0a-…",
"actor": { "id": "d0a2c7e1-…", "email": "editor@example.com" },
"data": {
"kind": "content_entry",
"id": "5ac9e8f2-…",
"contentType": "article",
"title": "Autumn catalogue"
}
}id is the delivery id and eventId the event id — the same two values as the
headers, frozen on the delivery row at enqueue time so the body and the headers
can never disagree. occurredAt is domain time, when the fact happened, not
when it was sent. data.kind and data.id name the aggregate; the rest of
data is what the event kind carries beyond the envelope — for an entry, its
content type and a title snapshot taken at the time of the event, so it
survives the record being deleted.
actor is null for a write made with an API token or by the system. Naming a
person who did not do it would be worse than saying nothing.
Verifying a delivery
The signature is HMAC-SHA256 over "{timestamp}.{raw body}", hex-encoded,
under the endpoint’s secret. The timestamp is inside the signed string, not
merely beside it: signing the body alone would let anyone who captured one
delivery replay it verbatim for as long as the secret lives. The v1= prefix
is what makes a future scheme change possible without breaking every receiver
on the day it ships.
Verify the raw bytes you received, not a re-serialised object — key order is not guaranteed to survive a round trip. Reject a timestamp older than five minutes, and compare in constant time.
import { createHmac, timingSafeEqual } from 'node:crypto';
/**
* @param {string} secret the endpoint's secret, `whsec_…`
* @param {string} header the X-Apograph-Signature header, `t=…,v1=…`
* @param {string} rawBody the request body exactly as received
*/
export function verifyApographSignature(secret, header, rawBody, tolerance = 300) {
const parts = Object.fromEntries(
header.split(',').map((part) => part.trim().split('='))
);
const timestamp = Number(parts.t);
const received = (parts.v1 ?? '').toLowerCase();
if (!Number.isInteger(timestamp) || !/^[0-9a-f]+$/.test(received)) {
return false;
}
const age = Math.abs(Math.floor(Date.now() / 1000) - timestamp);
if (age > tolerance) return false;
const expected = createHmac('sha256', secret)
.update(`${timestamp}.${rawBody}`)
.digest('hex');
if (expected.length !== received.length) return false;
return timingSafeEqual(Buffer.from(expected), Buffer.from(received));
}If the receiver is a Node project, @apograph/webhooks-domain exports the same
check as verifySignature(secret, header, rawBody) — shipped so the documented
snippet and the CMS’s own tests verify with the code that signed, because a
verifier written twice is a verifier that disagrees with itself eventually.
The secret is whsec_ followed by 32 random bytes as base64url. It is returned
in exactly two responses — the create and the rotate — and never again: every
other read returns secretHint, the last four characters, which is enough to
tell “the one I rotated to” from “the one I replaced” and far too little to
shorten a brute force.
Rotation takes effect immediately, queued deliveries included
A new secret signs everything sent after the rotate, including rows that were already waiting. Put the new secret in the receiver before rotating, or the deliveries in between are refused by your own verifier and retried against it until they die.
The retry ladder
A delivery gets six attempts. The wait after each failed one:
| After attempt | Wait |
|---|---|
| 1 | 10 seconds |
| 2 | 1 minute |
| 3 | 5 minutes |
| 4 | 30 minutes |
| 5 | 2 hours |
| 6 | given up — dead |
Roughly nine hours end to end: long enough to cover a receiver’s overnight outage, short enough that a delivery does not linger for days pretending it might still land. Every wait is jittered by ±20 % so a receiver coming back after an outage is not hit by the whole accumulated backlog on one tick.
What one attempt’s response means:
| Response | Verdict |
|---|---|
2xx | succeeded. Terminal |
408, 429 | Retry — they describe a moment, not the request. A Retry-After is honoured, capped at one hour, so a receiver cannot park the delivery |
Any other 4xx | dead, immediately. A rejected body or a wrong path is not going to be accepted on the sixth attempt, and hammering a receiver that already said no for nine hours is what gets a sender blocklisted |
3xx | Redirects are not followed. Reported as a failed attempt and retried — following one is the simplest way to be walked to an internal address after the first hop passed |
5xx | Retry |
| No status — timeout, refused connection, unresolvable host, rejected address | Retry, the same as a 5xx |
Twenty consecutive dead deliveries switch the endpoint off, with the reason shown in the admin. Without a ceiling, a staging URL torn down months ago keeps generating six requests per save for the life of the deployment. Re-enabling it by hand clears the counter. A test ping never counts.
Delivery states
| Status | Meaning |
|---|---|
pending | Queued, waiting for a worker |
delivering | Claimed; the request is in flight. A row stuck here past claimTimeoutMs is reclaimed on the assumption the worker died |
succeeded | A 2xx came back. Terminal |
failed | The last attempt failed and another is scheduled — nextAttemptAt says when |
dead | Given up on: attempts exhausted, a fatal response, or the endpoint was deleted or disabled before it was sent. Terminal |
Completed deliveries — succeeded and dead — are pruned once they are older
than retentionDays, at most once an hour. The log is the only table here that
nothing else prunes, and without this it grows at the rate content is edited.
The URL policy
A webhook is, precisely, “the server makes a request to an address a user typed”. That is the shape of every SSRF, so the policy is part of the domain rather than a check bolted onto the HTTP client, and the same function runs in two places: on the create and update form, so a bad URL is refused in the dialog with a message worth reading, and in the worker on every send, so a hostname re-pointed at a private address afterwards is refused on the way out.
By default a URL must be https://, carry no username or password (the
signature is how a receiver authenticates the sender, and credentials in the
URL would be sent on every retry and shown in the log), and resolve to a public
address. Loopback, link-local — including the cloud metadata endpoint at
169.254.169.254 — RFC 1918, CGNAT, the test networks, multicast and reserved
space are all refused, IPv4-mapped IPv6 included.
The address check runs after DNS resolution and before the socket opens, in a custom lookup on the HTTP agent. A name that passes a pre-flight check and then resolves to something private — DNS rebinding — never gets a connection.
allowInsecureUrls and allowPrivateNetworks widen that, and both are off by
default. The second is for a self-hosted install whose receiver genuinely sits
in the same cluster; an internet-facing deployment must not have it.
Custom headers are allowed — an Authorization for the receiver’s own gate is
the usual one — but none may start with X-Apograph-, and Host,
Content-Type, Content-Length, Transfer-Encoding, Connection and
User-Agent are reserved: a configured header must never be able to make a
delivery claim to be something it is not.
The admin
Four routes in the global directory group, beside API tokens, all gated on
webhooks:read:
| Route | What it is |
|---|---|
/webhooks | Every endpoint with its filters, enabled state and the outcome of its most recent delivery |
/webhooks/new | The editor. The secret is shown once, on save |
/webhooks/:id | The endpoint, its delivery log — filterable by state and event kind, newest first — and the actions |
/webhooks/:id/edit | The same editor, over an existing endpoint |
From the detail page: Send test posts a ping and shows the status, the
duration and the first couple of kilobytes of the response; Rotate secret
mints a new one and shows it once; and each row of the log opens to the frozen
request body and the response snippet, with Redeliver queueing the same
event again — new delivery id, same event id, so a receiver that deduplicates
still recognises the repeat.
Routes
All under /api/webhooks, global rather than workspace-scoped, and every
write carries the origin guard. The full table, with what each needs, is on the
endpoint reference.
A rejected URL answers 422, not 400: the request was well-formed and the
value was refused by policy. A delivery id belonging to another endpoint answers
404 — the log is not a way to enumerate deliveries across endpoints.
Permissions
webhooks:read and webhooks:manage are both administrator-only — even the
read, which is stricter than the alarms and segments splits, and deliberately
so. An endpoint is not scoped to a workspace, it reaches across every workspace
it names, and its delivery log records where this installation talks to on the
network. That is infrastructure configuration in the same family as an API
token, not something an editor is already working on. Redelivering needs
manage, not read: it causes an outgoing request to someone else’s system,
which is a write however it is spelled.
Limits
What this is not
- Not a message bus. Delivery is at-least-once and unordered. A receiver
gets the same event twice under a redelivery, a worker crash or an outbox
retry, and can get
entry.updatedafterentry.publishedfor the same record. Deduplicate onX-Apograph-Event-Id, and treat every delivery as “go and look”, never as the state. - Entry events and
pingonly. No media, transfer, account or workspace events, and no custom kinds from your own plugin. - No payload shaping. The body carries references; there is no option to inline field values, and no transformation or templating.
- No scheduled publishing to fire one at a future time, and no replay of history: an endpoint created today hears about tomorrow.
- One process sends unless you tell it not to. The worker is an in-process
interval, not a separate service;
deliveryIntervalMs: 0is how you make a node enqueue only.
See reacting to changes for the receiver’s side: when to poll instead, a receiver that verifies and deduplicates, and the cache-invalidation and rebuild patterns that follow.