Artifacts & Extraction¶

Every scraping job produces one or more artifacts — the outputs you actually consume. This page lists the artifact types, the manifest that describes them, how to download them, and how to pull structured data out of a page with extraction hooks.

Artifact types¶

Which artifacts a job produces depends on its artifact_options and tier.

Response metadata — HTTP status, response headers, and timing — is not a separate artifact. It lives in the manifest (see The manifest).

The manifest¶

Each job emits a manifest: the forensics record for the run. It describes every artifact produced (ids, types, sizes, integrity hashes) and, at the top level, captures the job's outcome, the target's HTTP response, and timing.

{
  "manifest_version": "1.0.0",
  "job_id": "3d7d1e6e-2b8e-47c2-8bbd-9c2a1a3f9b10",
  "engine": "stealth",
  "target_url": "https://example.com",
  "final_url": "https://example.com/",
  "status": "completed",
  "outcome": "success",
  "response": {
    "http_status": 200,
    "headers": { "content-type": "text/html; charset=utf-8" }
  },
  "timing": {
    "started_at": "2026-06-06T12:00:03Z",
    "finished_at": "2026-06-06T12:00:05Z",
    "duration_ms": 1250
  },
  "credits": { "weight": 30, "charged": true, "amount": 30 },
  "artifacts": [
    {
      "artifact_id": "7d6cbb9f-2f31-4e9c-8c3a-5e9b0f2a14e1",
      "artifact_type": "html",
      "mime_type": "text/html",
      "checksum_sha256": "…",
      "byte_size": 18342,
      "created_at": "2026-06-06T12:00:05Z"
    }
  ]
}

The same artifact list is returned inline on the job object (GET /api/v1/jobs/{jobId}) and, in the SDK, as Job.artifacts.

Status vs. outcome¶

The manifest separates two things that are easy to conflate:

A blocked job has status: "completed" (it ran) but outcome: "blocked", plus a structured block:

{
  "status": "completed",
  "outcome": "blocked",
  "block": { "blocked": true, "signal": "anti_bot_challenge", "http_status": 403 },
  "response": { "http_status": 403, "headers": { … } },
  "credits": { "weight": 30, "charged": false, "amount": 0 }
}

Response headers in the manifest are sanitized: cookies, set-cookie, authorization, and similar credential-bearing headers are redacted.

Downloading artifacts¶

Artifacts are never public. You exchange an artifact_id for a short-lived presigned URL, then fetch the bytes from that URL.

job = client.jobs.get("JOB_ID")
art = next(a for a in job.artifacts if a.artifact_type == "html")

html = client.artifacts.download_text(art.artifact_id)

# Exchange the id for a presigned URL (optionally set ttl_seconds, 60–86400)
curl "https://api.scrapenest.com/api/v1/artifacts/ARTIFACT_ID/download?ttl_seconds=600" \
  -H "X-API-Key: sn_live_..."
# → {"download_url": "https://...", "expires_at": "2026-06-06T12:15:05Z"}

curl -L "PRESIGNED_DOWNLOAD_URL" -o result.html

Integrity: every artifact carries a checksum_sha256. Verify it after download if you need tamper-evidence.

Extraction hooks¶

Instead of downloading HTML and parsing it yourself, let ScrapeNest extract structured data during the job. Add extraction.hooks to the payload and set artifact_options.include_extraction: true. Results come back as a json artifact.

A hook has an id, a type, and type-specific fields:

"extraction": {
  "hooks": [
    {"hook_id": "title", "type": "css", "selector": "h1"},
    {"hook_id": "image", "type": "css", "selector": "img.hero", "attribute": "src"},
    {"hook_id": "skus", "type": "regex", "pattern": "SKU-(\\d+)", "group": 1, "all_matches": true},
    {"hook_id": "price", "type": "jsonpath", "path": "$.product.price"}
  ]
}

Hook types¶

{"hook_id": "links", "type": "css", "selector": "a.product", "attribute": "href", "all_matches": true}

{"hook_id": "ids", "type": "regex", "pattern": "id=(\\d+)", "group": 1, "all_matches": true}

{"hook_id": "names", "type": "jsonpath", "path": "$.data[*].name"}

Result format¶

The extraction artifact contains one entry per hook:

{
  "hooks": [
    {
      "hook_id": "title",
      "type": "css",
      "status": "succeeded",
      "duration_ms": 4,
      "values": ["Example Domain"]
    },
    {
      "hook_id": "price",
      "type": "jsonpath",
      "status": "failed",
      "duration_ms": 1,
      "values": [],
      "error": "no_match"
    }
  ]
}

Limits¶

A failing or empty hook never fails the job — check each hook's status in the result.

Next steps¶

• Extract structured data — extraction hooks in a full recipe.
• Job Parameters — artifact_options and extraction in context.
• Data Retention & Holds — how long artifacts live.