Verifying Webhook Signatures¶

To ensure that the webhook events you receive are truly from ScrapeNest and haven't been tampered with, you must verify the request signature. ScrapeNest uses a Svix-style signing scheme.

Headers¶

Every webhook request includes the following mandatory headers:

• Svix-Id: A unique identifier for the message.
• Svix-Timestamp: The Unix timestamp (in seconds) when the message was sent.
• Svix-Signature: One or more space-separated v1,<base64> HMAC-SHA256 signatures.

Verification Steps¶

1. Validate the Timestamp: Prevent replay attacks by verifying that the Svix-Timestamp is within a 5-minute window of your current system time.
2. Construct the Signed Content: Concatenate the Svix-Id, the Svix-Timestamp, and the raw unparsed request body, separated by dots:

   msg_id.timestamp.payload
   

3. Compute the HMAC-SHA256: Generate the HMAC-SHA256 of the signed content, keyed on your Webhook Secret (the full whsec_... string, found in the Customer Console), then base64-encode the result.
4. Compare Signatures: Use a constant-time comparison to compare your computed v1,<base64> value against each signature in the Svix-Signature header.

Code Examples¶

Node.js (Express)¶

Use express.raw() to ensure you have the original, unparsed body for signature verification.

const crypto = require('crypto');

app.post('/webhooks', express.raw({type: 'application/json'}), (req, res) => {
    const payload = req.body.toString();
    const headers = req.headers;

    const svix_id = headers["svix-id"];
    const svix_timestamp = headers["svix-timestamp"];
    const svix_signature = headers["svix-signature"];

    // 1. Validate Timestamp
    const now = Math.floor(Date.now() / 1000);
    if (Math.abs(now - parseInt(svix_timestamp)) > 300) {
        return res.status(400).send("Expired timestamp");
    }

    // 2. Compute Signature (key = the full whsec_ secret, verbatim)
    const signed_content = `${svix_id}.${svix_timestamp}.${payload}`;
    const secret = "whsec_YOUR_SECRET";

    const hmac = crypto.createHmac('sha256', secret);
    const signature = hmac.update(signed_content).digest('base64');

    // 3. Constant-time Comparison
    const expected = `v1,${signature}`;
    const passed = svix_signature.split(' ').some(sig => {
        if (sig.length !== expected.length) return false;
        return crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected));
    });

    if (!passed) return res.status(400).send("Invalid signature");

    // Process valid event...
    res.status(204).send();
});

Python (Flask)¶

import hmac
import hashlib
import base64
import time
from flask import Flask, request

app = Flask(__name__)
SECRET = "whsec_YOUR_SECRET"

@app.route('/webhooks', methods=['POST'])
def handle_webhook():
    svix_id = request.headers.get("Svix-Id")
    svix_timestamp = request.headers.get("Svix-Timestamp")
    svix_signature = request.headers.get("Svix-Signature")
    payload = request.get_data(as_text=True)

    # 1. Validate Timestamp
    now = int(time.time())
    if abs(now - int(svix_timestamp)) > 300:
        return "Expired timestamp", 400

    # 2. Compute Signature (key = the full whsec_ secret, verbatim)
    signed_content = f"{svix_id}.{svix_timestamp}.{payload}"

    computed_signature = hmac.new(
        SECRET.encode(),
        signed_content.encode(),
        hashlib.sha256
    ).digest()

    expected_signature = f"v1,{base64.b64encode(computed_signature).decode()}"

    # 3. Constant-time Comparison
    signatures = svix_signature.split(' ')
    if not any(hmac.compare_digest(expected_signature, sig) for sig in signatures):
        return "Invalid signature", 400

    return "", 204