Skip to content

Anti-Blocking & Proxy Guard

Protected targets often check IP reputation, TLS fingerprints, browser fingerprints, request pacing, geolocation, resource loading patterns, and interaction timing. ScrapeNest exposes those controls through worker tiers, Proxy Guard, and Stealth jobs.

Proxy Routing

Proxy Guard routes jobs through managed residential and datacenter pools.

  • Rotation: distribute requests across proxy exits instead of reusing one IP for every job.
  • Geotargeting: originate requests from target markets such as US, FR, DE, or JP.
  • Session stickiness: keep all requests in one browser job on the same exit IP.
  • Custom egress: on the Stealth tier, route through your own proxy instead of the managed pool.

Custom egress (bring your own proxy)

On the Stealth tier you can pass a proxy object to send a job's traffic through your own proxy - useful when you need a specific exit IP or your own proxy contracts.

Key points:

  • Chained, not a bypass: traffic still flows through ScrapeNest's Proxy Guard first, then out through your proxy. Proxy Guard remains the egress boundary.
  • Stealth only: proxy is rejected on the Light and Standard tiers.
  • Public hosts only: proxy.server must use http or https, include an explicit port, and resolve to a public address. Private or reserved addresses are rejected.
  • Credentials: supply username/password (or embed them in the server URL). They are used only to reach your proxy and are never returned or logged.
  • Billing is unchanged: a job that uses your proxy costs the same as any Stealth job. You cover the egress, ScrapeNest covers the scrape.
  • Confirm the exit: the job manifest's egress block reports mode: custom and the proxy host when your proxy was used.
{
  "job_type": "stealth",
  "target_url": "https://protected.example.com",
  "proxy": {
    "server": "http://proxy.example.com:8080",
    "username": "user",
    "password": "secret",
    "bypass": "*.example.com"
  }
}

Fingerprint Management

ScrapeNest employs different levels of fingerprinting based on your selected Worker Tier.

TLS & HTTP Impersonation (Light Tier)

light jobs use TLS and HTTP settings that look like modern browser traffic instead of a default server-side HTTP client.

Browser Fingerprinting (Standard Tier)

standard jobs run in a browser context and can control viewport, locale, timezone, User-Agent, headers, resource loading, and interactions. Those browser-profile fields are Standard-tier controls - see the Stealth section below.

Hardened Stealth Engine (Stealth Tier)

stealth jobs add deeper anti-detection controls for targets that block standard automation:

  • OS profile selection with os_name: windows, macos, or linux.
  • Hardened browser fingerprint surfaces including Canvas, WebGL, AudioContext, and WebRTC.
  • Managed proxy pool integration with session stickiness.
  • Optional helper extensions such as ublock and isdcac.
  • Resource-blocking options to reduce noisy third-party scripts.
  • A self-consistent browser profile - viewport, locale, timezone and User-Agent - supplied by the fingerprint. Because overriding one facet while the rest stays put would contradict it (a classic detection signal), none of viewport, locale, timezone_id or user_agent is accepted on this tier; each returns 422 invalid_<field>_tier. Choose the profile with os_name, or use Standard when you need exact values.

Tuning for Higher Success Rates

  1. Start with the cheapest tier that works: light for direct responses, standard for rendering, stealth for protected targets.
  2. Match the visitor profile: on Stealth, pick it with os_name and geolocation - the viewport, locale, timezone and User-Agent all come from the hardened fingerprint as one consistent set, which is why the tier rejects per-job overrides (invalid_viewport_tier, invalid_locale_tier, invalid_timezone_id_tier, invalid_user_agent_tier) instead of letting you make the profile self-contradictory. On Standard you set viewport, locale, timezone_id and user_agent yourself.
  3. Use waits and actions: set wait_until, add explicit wait actions, scroll lazy-loaded pages, and click overlays when needed.
  4. Reduce noise: block images, media, fonts, or stylesheets when they are not required for the output.
  5. Pace per target: use rate limits, job concurrency, and idempotency tokens for large crawls.
  6. Inspect diagnostics: enable metadata, HAR, console logs, and screenshots when tuning a difficult target.

See also