Job Parameters¶

Every scraping job is a single JSON payload sent to POST /v1/jobs (or passed as keyword arguments to the Python SDK). This page is the complete reference for that payload, grouped by area, with types, defaults, and which worker tier honors each option.

Core (all tiers)¶

client.scrape(
    job_type="light",
    target_url="https://example.com",
    timeout_ms=45000,
    tags=["env:prod"],
)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{"job_type": "light", "target_url": "https://example.com", "timeout_ms": 45000, "tags": ["env:prod"]}'

HTTP options (Light tier)¶

These apply to job_type: "light", which uses a high-performance HTTP engine with TLS impersonation.

retry_policy shape:

client.scrape(
    job_type="light",
    target_url="https://api.example.com/products",
    method="POST",
    headers={"Accept": "application/json"},
    body='{"query": "widgets"}',
    retry_policy={"max_attempts": 3, "retry_on_status": [429, 500, 502], "retry_methods": ["GET", "POST"]},
)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "light",
    "target_url": "https://api.example.com/products",
    "method": "POST",
    "headers": {"Accept": "application/json"},
    "body": "{\"query\": \"widgets\"}",
    "retry_policy": {"max_attempts": 3, "retry_on_status": [429, 500, 502], "retry_methods": ["GET", "POST"]}
  }'

Browser options (Standard & Stealth tiers)¶

These apply to job_type: "standard" and job_type: "stealth", which render the page in a real browser.

client.scrape(
    job_type="standard",
    target_url="https://example.com/dashboard",
    wait_until="networkidle",
    viewport={"width": 1920, "height": 1080},
    locale="fr-FR",
    timezone_id="Europe/Paris",
)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "standard",
    "target_url": "https://example.com/dashboard",
    "wait_until": "networkidle",
    "viewport": {"width": 1920, "height": 1080},
    "locale": "fr-FR",
    "timezone_id": "Europe/Paris"
  }'

Actions¶

actions is an ordered list of interactions executed after the page loads - clicks, typing, scrolling, and waits. Use them to dismiss banners, expand content, or drive a multi-step flow before capture.

"actions": [
  {"type": "click", "selector": "#accept-cookies"},
  {"type": "fill", "selector": "input[name=q]", "value": "widgets"},
  {"type": "scroll", "direction": "down"},
  {"type": "wait", "timeout_ms": 1000}
]

See Scrape a JavaScript SPA for worked examples.

Stealth & anti-blocking options (Stealth tier)¶

These apply to job_type: "stealth", the hardened browser tier for protected targets. See Anti-Blocking and Work with protected targets.

The job manifest records the effective egress under egress (mode: managed or mode: custom with the proxy host, never credentials), so you can confirm which exit a job used.

client.scrape(
    job_type="stealth",
    target_url="https://protected.example.com",
    os_name="macos",
    browser_extensions=["ublock", "isdcac"],
)

client.scrape(
    job_type="stealth",
    target_url="https://protected.example.com",
    proxy={
        "server": "http://proxy.example.com:8080",
        "username": "user",
        "password": "secret",
    },
)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "stealth",
    "target_url": "https://protected.example.com",
    "proxy": {"server": "http://proxy.example.com:8080", "username": "user", "password": "secret"}
  }'

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "stealth",
    "target_url": "https://protected.example.com",
    "os_name": "macos",
    "browser_extensions": ["ublock", "isdcac"]
  }'

Artifact options¶

artifact_options controls which outputs a job produces and how the browser loads the page. Set only what you need - fewer artifacts means faster, cheaper jobs. See Artifacts & Extraction.

Response metadata (HTTP status, response headers, timing) is always recorded in the job manifest - there is no separate metadata option or artifact.

client.scrape(
    job_type="stealth",
    target_url="https://example.com",
    artifact_options={
        "include_html": True,
        "include_screenshot": True,
        "block_images": True,
        "block_fonts": True,
    },
)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: sn_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "stealth",
    "target_url": "https://example.com",
    "artifact_options": {"include_html": true, "include_screenshot": true, "block_images": true, "block_fonts": true}
  }'

Extraction¶

extraction.hooks pulls structured data out of the page using CSS selectors, regular expressions, or JSONPath - no post-processing on your side. This is documented in full, with limits and the result format, in Artifacts & Extraction → Extraction hooks.

"extraction": {
  "hooks": [
    {"hook_id": "title", "type": "css", "selector": "h1"},
    {"hook_id": "price", "type": "css", "selector": ".price", "attribute": "data-amount"},
    {"hook_id": "sku", "type": "regex", "pattern": "SKU-(\\d+)", "all_matches": true}
  ]
}

Validation & errors¶

The API validates the payload before accepting a job. Invalid parameters return 422 with a machine-readable error code naming the offending field - for example invalid_wait_until, invalid_viewport_width, invalid_method, invalid_timeout_ms. Fix the parameter and resubmit; these are not retryable. See Errors & Retries.

Enforced constraints worth remembering:

• timeout_ms and navigation_timeout_ms: 1000–300000.
• max_redirects: 0–20.
• viewport.width: 320–7680; viewport.height: 240–4320. Standard tier only - viewport on Stealth returns invalid_viewport_tier.
• wait_until ∈ {load, domcontentloaded, networkidle, commit}.
• method ∈ {GET, POST, PUT, PATCH, DELETE, HEAD}.

Next steps¶

• Artifacts & Extraction - what comes back and how to read it.
• Worker Tiers - pick the right engine.
• Guides - these parameters in real recipes.