Webhook Consumers Need a Replay Contract

A billing webhook arrives, the consumer updates an account, and its response disappears before reaching the provider. The provider retries. If the handler applies the same change twice, successful delivery has become a correctness bug. If the handler rejects every repeated request before recovering unfinished work, a crash can permanently lose the update.

A useful webhook contract separates authenticity, receipt, and business processing. A signature establishes something about the request bytes and the signing secret. An inbox records which delivery was accepted. A business transaction establishes whether the intended effect happened. These facts need separate evidence even when a small application implements them in one service.

Verify the bytes you received

For an HMAC-based provider, compute the signature over the exact raw request body. Parsing JSON and serializing it again can change whitespace, key order, or escaping. The parsed object may look identical while the byte sequence no longer matches the signature. Enforce a body limit while reading the request, before buffering an unbounded payload.

Use the provider's documented algorithm and header format, and compare signatures with a timing-resistant comparison function. Keep signing secrets outside source code. Rotation needs a bounded transition in which the accepted key set is explicit; retaining every old secret forever expands the period in which a leaked key remains useful.

The example below uses GitHub's sha256= header convention and a public fixture secret. Its SQLite inbox is local and in memory, so it demonstrates transaction and deduplication behavior, not crash durability. A deployed inbox needs persistent storage and a durability configuration that matches its acknowledgment promise.

Store an identity and a fingerprint

import hashlib
import hmac
import sqlite3

secret = b"public-demo-fixture-not-a-real-secret"
db = sqlite3.connect(":memory:")
db.execute("""CREATE TABLE inbox (
    delivery_id TEXT PRIMARY KEY NOT NULL,
    digest TEXT NOT NULL, payload BLOB NOT NULL
)""")


def signature(raw):
    return "sha256=" + hmac.new(secret, raw, hashlib.sha256).hexdigest()


def receive(delivery_id, raw, supplied_signature):
    if not isinstance(delivery_id, str) or not delivery_id.strip():
        raise ValueError("delivery identity required")
    if len(raw) > 1024 * 1024:
        raise ValueError("body too large")
    if not isinstance(supplied_signature, str):
        raise ValueError("signature required")
    if not hmac.compare_digest(signature(raw), supplied_signature):
        raise ValueError("bad signature")
    digest = hashlib.sha256(raw).hexdigest()
    with db:
        inserted = db.execute(
            "INSERT OR IGNORE INTO inbox VALUES (?, ?, ?)",
            (delivery_id, digest, raw),
        ).rowcount
        if not inserted:
            old = db.execute(
                "SELECT digest FROM inbox WHERE delivery_id = ?",
                (delivery_id,),
            ).fetchone()[0]
            if old != digest:
                raise ValueError("identity reused with another payload")
    return "stored" if inserted else "duplicate"


body = b'{"order":"A7","state":"paid"}'
sig = signature(body)
assert receive("delivery-1", body, sig) == "stored"
assert receive("delivery-1", body, sig) == "duplicate"
for raw, supplied in ((body + b" ", sig), (b"{}", signature(b"{}"))):
    try:
        receive("delivery-1", raw, supplied)
    except ValueError:
        pass
    else:
        raise AssertionError("tamper or identity conflict accepted")
assert db.execute("SELECT count(*) FROM inbox").fetchone()[0] == 1

The first delivery stores the original bytes. The next identical delivery returns a duplicate result without adding another row. Reusing the delivery identity with a different signed body raises a conflict. That conflict should become an operational signal, not an instruction to overwrite history with the newest payload.

Validate identity fields before using them as keys, and scope the key to the provider or installation when their identifiers are not globally unique. A database constraint should enforce the uniqueness rule. A check followed by an unrelated insert leaves a race between concurrent deliveries; the example lets the insert arbitrate first and then inspects the stored fingerprint.

Be precise about replay protection

A valid body signature does not necessarily prove freshness. In the convention shown here, the body is signed; the delivery identifier is not part of that signature calculation. Someone able to replay captured signed bytes could change an unsigned identifier. The inbox protects normal retries using stable provider identities, but it is not a general defense against every malicious replay.

Some providers sign a timestamp and define an acceptance window. Follow that protocol exactly, including clock-skew tolerance and which deliveries may be retried later. When no signed freshness mechanism exists, rely on the documented provider contract, protected transport, restricted secret access, and business-level uniqueness where available. Do not invent a timestamp field that the sender never authenticated.

Two distinct deliveries can also describe the same business fact. An invoice identifier and a monotonic state transition may be more useful than a delivery identifier for preventing a second credit. Conversely, two legitimate events can have identical bodies. Deduplicating only by body hash can discard valid work. The fingerprint in this example detects identity misuse; it is not the identity itself.

Acknowledge receipt, then finish the work

Return the provider's successful acknowledgment only after the inbox transaction commits. If the process crashes before the response, a repeated delivery can discover the existing row. A worker should process accepted rows and track pending, retryable, and terminal outcomes so that duplicate receipt does not suppress unfinished work.

When the business effect lives in the same database, apply it and mark the inbox item processed in one transaction. If processing must call an external service, use that service's idempotency mechanism or record an outbound obligation with an outbox. Marking an inbox row processed before an uncertain remote call creates a loss window; marking it afterward creates a replay window that needs an explicit solution.

Event order is another independent concern. A later account state may arrive before an earlier one. Prefer a provider version, a state transition rule, or a fresh authoritative read when the API supports it. Delivery timestamps alone do not establish a total business order across concurrent events.

Retention is part of the guarantee

An inbox retained for seven days cannot promise delivery deduplication on day eight. Choose retention from the provider's retry and manual replay windows, then document what happens outside that period. Long-lived business identities may need longer retention than raw request bodies, which can contain sensitive data.

Monitor signature failures, identity conflicts, oldest pending age, processing retries, and terminal failures. Test a crash after durable receipt and before processing, then a crash after the business commit and before acknowledgment. A healthy webhook pipeline can explain each delivery's current state without treating another retry as a surprise.

Sources and further reading

• GitHub: validating webhook deliveries
• Python HMAC comparison
• SQLite transaction semantics