Change Detection¶

Turn any monitor into a watcher. When the page you monitor - or the part you select - changes between runs, ScrapeNest notifies you by webhook and email and stores a before/after diff. Use it for price and stock monitoring, competitor and content changes, terms-of-service updates, and regulatory pages.

Change detection is an optional detection block on a monitor. Every run of that monitor is fingerprinted, compared to the previous version, and (on a real change) diffed and notified. A watched run is billed exactly like any other run - there is no separate charge for detection.

How it works¶

• The first evaluated run captures a reference version (the baseline) and sends a schedule.monitor_activated confirmation - not a change alert.
• Each later run is compared to the previous one. Only runs that actually delivered content are evaluated; a blocked or failed run is never treated as a change.
• The fingerprint is computed over normalized content (whitespace collapsed, scripts/styles and your ignored selectors removed), so rotating ads, timestamps, and session tokens do not cause false alerts.
• On a change, ScrapeNest records it in the change history, generates a diff, and - if the change clears your thresholds - sends schedule.change_detected by webhook and email.

Plan limits¶

Enabling detection on a plan that does not include it (or a mode it does not allow) is rejected at creation time.

The detection block¶

Modes¶

extract mode needs extraction hooks configured on the job. If none produced data, the detection health shows selector_missing (it needs attention).

Enable detection¶

from scrapenest import ScrapeNestClient

client = ScrapeNestClient(api_key="sn_live_...", base_url="https://api.scrapenest.com")

monitor = client.monitors.create(
    name="competitor-pricing",
    cron="0 * * * *",
    job_type="standard",
    target_url="https://competitor.example/pricing",
    detection={
        "enabled": True,
        "mode": "selector",
        "selector": ".price, #stock",
        "ignore_selectors": [".ads", "time"],
        "min_change_ratio": 0.05,
        "notify": {"webhook": True, "email": ["alerts@acme.eu"]},
    },
)
print(monitor.detection)   # config + health (status, last_checked_at, last_change_at)

curl -X POST "https://api.scrapenest.com/api/v1/monitors" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "competitor-pricing",
    "cron": "0 * * * *",
    "job_type": "standard",
    "target_url": "https://competitor.example/pricing",
    "detection": {
      "enabled": true,
      "mode": "selector",
      "selector": ".price, #stock",
      "ignore_selectors": [".ads", "time"],
      "min_change_ratio": 0.05,
      "notify": {"webhook": true, "email": ["alerts@acme.eu"]}
    }
  }'

Detection health¶

GET /api/v1/monitors/{id} returns the detection block including a health object so you can answer "is my monitor working?" without guessing:

• status - ok, or selector_missing when your selector no longer matches the page (the monitor needs attention - the page structure likely changed).
• last_checked_at - when the monitor last evaluated a run.
• last_change_at - when it last detected a change.

Review changes¶

GET /api/v1/monitors/{id}/changes returns the change history, newest first.

for change in client.monitors.iter_changes(monitor.id):
    pct = f"{change.change_ratio * 100:.1f}%" if change.change_ratio is not None else "n/a"
    print(change.detected_at, "changed", pct, "notified:", change.notified_channels)

curl "https://api.scrapenest.com/api/v1/monitors/MONITOR_ID/changes" \
  -H "X-API-Key: sn_live_..."

Each change carries old_fingerprint, new_fingerprint, change_ratio, has_diff, and the notified_channels used (empty when a change was recorded but its notification was debounced).

Updating and turning it off¶

On update, omit detection to leave it unchanged. Changing the mode or selector resets the baseline (the next run re-captures a reference version). To turn detection off while keeping the monitor running, send "detection": {"enabled": false}.

Notifications¶

Subscribe to schedule.change_detected and schedule.monitor_activated, and/or list emails in notify.email. The change notification includes the change ratio and a short-lived signed link to the diff.

Digest emails¶

For high-frequency monitors, set notify.digest to daily or weekly to receive one summary email per period instead of one email per change. Webhooks still fire in real time on every change; only the email is batched.

"notify": { "webhook": true, "email": ["ops@acme.eu"], "digest": "daily" }

Viewing the diff¶

Every detected change with a stored diff exposes it at GET /api/v1/monitors/{id}/changes/{changeId}/diff (also rendered in the console). Lines prefixed with - were removed and + were added. For extract/json monitors this is a field-level diff:

  {
-   "price": "EUR 49"
+   "price": "EUR 39"
  }