Rate Limits & Quotas¶

ScrapeNest enforces limits at two levels — your organization (plan-wide) and each API key. Together they protect platform stability and your budget. You set the per-key limits when you create or update a key.

Organization limits¶

Plan-wide limits apply across every key in your organization.

Tier	Requests per Minute (RPM)	Requests per Hour (RPH)
Trial	60	1,000
Standard	600	10,000
Enterprise	3,000	60,000

Concurrency (how many jobs run at once) and your monthly credit pool are also plan-wide — see Billing & Usage.

Per-key limits¶

When creating an API key (Console → Developer → API Keys, or POST /api/v1/keys), you can scope tighter limits to that specific key. This is ideal for isolating a noisy integration or a third party.

Setting	Description
`rate_limit_rpm`	Requests per minute allowed for this key.
`daily_quota`	Maximum requests per day for this key.
`max_concurrent_jobs`	Maximum jobs this key may run simultaneously.
`enforcement_mode`	`block` rejects requests over the limit with `429`; `observe` only meters and logs them (useful for sizing limits before enforcing).

curl -X POST "https://api.scrapenest.com/api/v1/keys" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nightly-batch",
    "scopes": ["jobs:write", "jobs:read"],
    "rate_limit_rpm": 120,
    "daily_quota": 50000,
    "max_concurrent_jobs": 10,
    "enforcement_mode": "block"
  }'

The effective limit on any request is the most restrictive of the org and key limits.

Handling 429s¶

When a request exceeds an enforced limit, the API returns 429 Too Many Requests. Inspect these headers:

X-RateLimit-Limit — the total request limit for the window.
X-RateLimit-Remaining — requests remaining in the current window.
X-RateLimit-Reset — when the window resets.
Retry-After — seconds to wait before retrying (when present).

Retry strategy¶

Back off exponentially on 429, preferring Retry-After/X-RateLimit-Reset over a fixed delay, and always pair retries with an idempotency_token. The Python SDK errors guide has a ready-made retry helper.

Pace large crawls

Most 429s come from bursting. When paginating or crawling, spread submissions and use max_concurrent_jobs to self-limit.