Quickstart¶

Go from zero to your first scraped page in under five minutes. We'll install the SDK, create an API key, submit a job, and read the result.

1. Get an API key¶

API keys authenticate machine requests.

1. Open the Customer Console and go to Settings → API Keys.
2. Click Create New Key.
3. Select the Scopes the key needs (at minimum jobs.create and jobs.read; add artifacts.read to download results).
4. (Optional) Set an IP Allowlist to restrict the key to your infrastructure's CIDR blocks.
5. Copy the key now - keys are stored as Argon2id hashes and cannot be retrieved again.

2. Install the SDK¶

pip install scrapenest-sdk

# No install needed - curl is enough to call the REST API.
export SCRAPENEST_API_KEY="sn_live_..."

3. Submit your first job¶

ScrapeNest offers three worker tiers. Start with Light - it's the fastest and cheapest, and it handles APIs and static pages.

from scrapenest import ScrapeNestClient

client = ScrapeNestClient(
    api_key="sn_live_...",
    base_url="https://api.scrapenest.com",
)

result = client.scrape(
    job_type="light",
    target_url="https://example.com",
)

print(result.status)          # "succeeded"
print(result.job_id)
print(result.artifact_count)

curl -X POST "https://api.scrapenest.com/v1/jobs" \
  -H "X-API-Key: $SCRAPENEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "light",
    "target_url": "https://example.com",
    "tags": ["env:prod", "type:catalog"]
  }'

scrape submits the job and blocks until it finishes (up to 120s). For long jobs or high throughput, use submit and webhooks instead.

Choosing a tier¶

client.scrape(job_type="light", target_url="https://example.com/api/products")

client.scrape(
    job_type="standard",
    target_url="https://example.com/dashboard",
    wait_until="networkidle",
)

client.scrape(
    job_type="stealth",
    target_url="https://protected.example.com",
    os_name="macos",
    browser_extensions=["ublock", "isdcac"],
)

See Worker Tiers for a full comparison and Job Parameters for every option.

4. Read the result¶

A finished job produces artifacts - rendered HTML, screenshots, extracted JSON, and metadata. List them on the job, then request a short-lived presigned URL for the bytes.

job = client.jobs.get(result.job_id)
for a in job.artifacts:
    print(a.artifact_type, a.artifact_id)

html_artifact = next(a for a in job.artifacts if a.artifact_type == "html")
html = client.artifacts.download_text(html_artifact.artifact_id)

# Fetch the job (include_download_urls returns artifact URLs inline)
curl "https://api.scrapenest.com/v1/jobs/JOB_ID?include_download_urls=true" \
  -H "X-API-Key: $SCRAPENEST_API_KEY"

# Download an artifact's bytes from its presigned URL
curl -L "PRESIGNED_DOWNLOAD_URL" -o result.html

Artifacts are governed by your organization's retention policy. You can place legal holds on critical data to prevent automated purging.

Next steps¶

• MCP Agents - give your CLI or web agent live-web tools through your ScrapeNest account.
• Python SDK - the full client reference (sync vs async, options, artifacts).
• Recipes - ready-made scrapers for well-known sites that return schema-validated JSON.
• Guides - copy-paste examples: extract data, screenshots, SPAs, anti-bot.
• Authentication - RBAC, scopes, and IP allowlisting.
• Webhooks - get notified instead of polling.