> ## Documentation Index > Fetch the complete documentation index at: https://brezelscraper.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Webhooks > Receive notifications when scraping jobs finish. Webhooks let BrezelScraper send an HTTP POST to your server when a job completes, fails, or is cancelled. You register a URL, and BrezelScraper calls it with a small JSON payload every time a job reaches a final state. ## Quick start 1. Go to **Dashboard > Integrations** and create a webhook. 2. Paste an HTTPS endpoint URL from your server or automation tool. 3. Copy the **signing secret** and store it securely. You will only see it once. 4. Create a scraping job. When the job finishes, BrezelScraper sends a POST to your URL. 5. Your endpoint receives the event, verifies the signature, and fetches results through the API. ## Event types BrezelScraper sends one of three event types: | Event type | When it fires | | --------------- | -------------------------------------------------------------- | | `job.completed` | The job finished successfully. Results are ready. | | `job.failed` | The job could not complete. Check `failure_reason` on the job. | | `job.cancelled` | The job was cancelled by the user or by a timeout. | ## Payload format Every webhook delivery is an HTTP POST with a JSON body: ```json theme={null} { "event_type": "job.completed", "job_id": "d8d8a24e-cef2-4b02-8396-abc290b6f299", "job_name": "Berlin cafes", "status": "completed", "result_count": 87, "created_at": "2026-04-10T12:00:00Z", "ended_at": "2026-04-10T12:05:30Z" } ``` | Field | Type | Description | | -------------- | ------- | ----------------------------------------------------------------- | | `event_type` | string | One of `job.completed`, `job.failed`, `job.cancelled` | | `job_id` | string | The job's UUID. Use this to fetch results from the API. | | `job_name` | string | The name you gave the job when you created it. | | `status` | string | The job's final status. | | `result_count` | integer | Number of places scraped. `0` if the job failed or found nothing. | | `created_at` | string | When the job was created (ISO 8601). | | `ended_at` | string | When the job reached its final status (ISO 8601). | The payload contains metadata only. To get the scraped data, call `GET /api/v1/jobs/{job_id}/results` after receiving the webhook. ## Headers Each delivery includes these headers: | Header | Example | Purpose | | --------------------- | ------------------------------- | ----------------------------------------------------- | | `Content-Type` | `application/json` | Always JSON. | | `User-Agent` | `BrezelScraper-Webhook/1.0` | Identifies the sender. | | `X-Webhook-Signature` | `t=1712956800,sha256=a1b2c3...` | Signed timestamp and HMAC-SHA256 of the request body. | | `X-Webhook-ID` | `019d837e-8e1c-...` | Unique ID for this delivery. Use it to deduplicate. | The signature header contains two parts separated by a comma: * `t=` is the timestamp when the delivery was sent * `sha256=` is the HMAC-SHA256 of `.` using your signing secret The timestamp is included in the signed payload, so it cannot be forged separately. ## Verify the signature Parse the `X-Webhook-Signature` header, extract the timestamp and signature hash, then compute the expected HMAC over `timestamp + "." + body`. ### Node.js ```javascript theme={null} const crypto = require('crypto'); function verifyWebhook(body, secret, signatureHeader) { const parts = Object.fromEntries( signatureHeader.split(',').map(p => p.split('=')) ); const timestamp = parts.t; const signature = parts.sha256; // Reject deliveries older than 5 minutes const age = Math.floor(Date.now() / 1000) - parseInt(timestamp, 10); if (age > 300) return false; const expected = crypto .createHmac('sha256', secret) .update(timestamp + '.' + body, 'utf8') .digest('hex'); return crypto.timingSafeEqual( Buffer.from(signature), Buffer.from(expected) ); } // In your handler: const isValid = verifyWebhook(rawBody, process.env.WEBHOOK_SECRET, req.headers['x-webhook-signature']); if (!isValid) { return res.status(401).send('Invalid signature'); } ``` ### Python ```python theme={null} import hmac import hashlib import time def verify_webhook(body: bytes, secret: str, signature_header: str) -> bool: parts = dict(p.split("=", 1) for p in signature_header.split(",")) timestamp = parts["t"] signature = parts["sha256"] # Reject deliveries older than 5 minutes if int(time.time()) - int(timestamp) > 300: return False expected = hmac.new( secret.encode(), f"{timestamp}.".encode() + body, hashlib.sha256 ).hexdigest() return hmac.compare_digest(signature, expected) ``` ### Go ```go theme={null} import ( "crypto/hmac" "crypto/sha256" "crypto/subtle" "encoding/hex" "strings" ) func verifyWebhook(body []byte, secret string, signatureHeader string) bool { parts := make(map[string]string) for _, p := range strings.Split(signatureHeader, ",") { kv := strings.SplitN(p, "=", 2) if len(kv) == 2 { parts[kv[0]] = kv[1] } } timestamp := parts["t"] signature := parts["sha256"] mac := hmac.New(sha256.New, []byte(secret)) mac.Write([]byte(timestamp)) mac.Write([]byte(".")) mac.Write(body) expected := hex.EncodeToString(mac.Sum(nil)) return subtle.ConstantTimeCompare([]byte(signature), []byte(expected)) == 1 } ``` Always use constant-time comparison (`timingSafeEqual` in Node.js, `compare_digest` in Python, `subtle.ConstantTimeCompare` in Go) instead of `===` or `==`. Regular string comparison can leak timing information that helps attackers forge signatures. The timestamp is cryptographically bound to the signature. Reject deliveries where `t` is more than 5 minutes old to prevent replay attacks. ## Retries If your endpoint returns a non-2xx status code or does not respond, BrezelScraper retries the delivery up to 5 times with exponential backoff and jitter: * Backoff: roughly `2^attempt` seconds, multiplied by a random jitter factor in `[0.5, 1.5)` * Cap: 1 hour between attempts | Attempt # | Approximate wait before retry | | --------- | ----------------------------- | | 1 → 2 | 1–3 seconds | | 2 → 3 | 2–6 seconds | | 3 → 4 | 4–12 seconds | | 4 → 5 | 8–24 seconds | After 5 failed attempts the delivery is marked as failed and not retried again. Each retry uses the same `X-Webhook-ID`, so you can deduplicate on that header if your endpoint received the same delivery more than once. ## Health states and the circuit breaker Every webhook configuration carries a `health_state`: | State | Meaning | | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | `healthy` | Default. Deliveries flow normally. | | `degraded` | Five consecutive deliveries have failed end-to-end (each after its full retry budget). No behavior change yet — just a signal that something is flapping. | | `disabled` | Ten consecutive deliveries have failed. The breaker is open; no new deliveries are attempted until you re-enable. | Each delivery has its own 5-retry budget with exponential backoff. The breaker counter only increments when a delivery has exhausted that budget — so `disabled` means "10 separate jobs failed end-to-end," not "10 individual HTTP attempts." A single successful (2xx) delivery anywhere in the streak resets `consecutive_failures` to 0 and restores `health_state` to `healthy`. ### Re-enabling a disabled webhook To re-enable after fixing your endpoint, `PATCH` the config with `{"reenable": true}`: ```bash theme={null} curl -X PATCH https://api.brezelscraper.com/api/v1/webhooks/{id} \ -H "Authorization: Bearer bscraper_your_key_here" \ -H "Content-Type: application/json" \ -d '{"reenable": true}' ``` You can mix `reenable` with `name` / `url` in the same request — the re-enable is applied first, so a re-enable failure won't half-apply your metadata change. ### Listing disabled webhooks `GET /api/v1/webhooks` returns every config you own, including `disabled` ones, so the UI can render a "Re-enable" CTA. Each entry now includes: ```json theme={null} { "health_state": "disabled", "consecutive_failures": 10, "disabled_at": "2026-05-19T17:42:11Z", "disabled_reason": "http_5xx" } ``` `disabled_reason` is one of: * `http_5xx` — your endpoint returned a 5xx * `http_4xx` — your endpoint returned a 4xx * `transport_error` — connection refused, TLS error, timeout, etc. It captures the failure mode that was active when the breaker tripped. ## Rate limits To protect your endpoint and prevent abuse, BrezelScraper limits webhook deliveries to: * **100 deliveries per hour** per user account * **50 deliveries per hour** per destination IP address If a delivery is rate-limited, it is automatically retried after the limit window resets. ## Requirements and limits | Setting | Value | | ------------------------------- | ---------------------- | | Protocol | HTTPS only | | Max active webhooks per account | 10 | | Webhook name length | 1 to 100 characters | | URL length | Up to 2,048 characters | | Response timeout | 10 seconds | | Max retry attempts | 5 | | Signing algorithm | HMAC-SHA256 | URLs pointing to private networks, localhost, or cloud metadata endpoints are rejected. ## Manage webhook configurations ### `POST /api/v1/webhooks` Create a new webhook. **Request body:** ```json theme={null} { "name": "My webhook", "url": "https://example.com/webhooks/brezel" } ``` **Response** (`201 Created`): ```json theme={null} { "id": "018efdf7-7fe1-7c42-b4e2-123456789abc", "name": "My webhook", "url": "https://example.com/webhooks/brezel", "secret": "fdc7992336902be147a38f36a24876828f4207d5820d430140c9602c04fbd7a5" } ``` The `secret` is your signing key. Store it securely. It is shown only at creation time. ### `GET /api/v1/webhooks` List all webhook configurations for your account, including revoked ones. **Response** (`200 OK`): ```json theme={null} [ { "id": "018efdf7-7fe1-7c42-b4e2-123456789abc", "name": "My webhook", "url": "https://example.com/webhooks/brezel", "verified_at": "2026-04-10T09:00:00Z", "created_at": "2026-04-10T09:00:00Z", "updated_at": "2026-04-10T09:00:00Z" } ] ``` `verified_at` is set after the first successful delivery to this webhook. This endpoint returns a plain array because each account is limited to 10 webhooks. No pagination is needed. ### `PATCH /api/v1/webhooks/{id}` Update a webhook's name, URL, or re-enable the circuit breaker. All fields are optional. ```json theme={null} { "name": "Renamed webhook", "url": "https://example.com/new-endpoint", "reenable": true } ``` `reenable: true` clears any `disabled` state and resets `consecutive_failures` to 0. Sending `reenable: true` on an already-healthy config is a no-op success (the call is idempotent). `reenable: false` is also a no-op — only the delivery worker can disable a config; users only ever re-enable. Returns `204 No Content` on success. ### `DELETE /api/v1/webhooks/{id}` Revoke a webhook. Revoked webhooks stop receiving deliveries. Returns `204 No Content` on success. ## Best practices * **Respond quickly.** Return a 200 status code as fast as possible. If you need to do heavy processing, queue the work and respond immediately. * **Verify every delivery.** Always check the `X-Webhook-Signature` header before acting on the payload. * **Deduplicate.** Use `X-Webhook-ID` to detect and skip duplicate deliveries. * **Fetch results from the API.** The webhook payload tells you a job finished. Call `GET /api/v1/jobs/{job_id}/results` to get the actual data. * **Handle all event types.** Your endpoint should return 200 for events it does not care about. Returning an error causes unnecessary retries.