Wayfern
Start building Sign in

Reduce residential proxy bandwidth

Cut residential bandwidth cost: start on the free datacenter exit, block images and media, reuse cached pages, and learn which calls Wayfern meters per byte.

Updated

Residential bandwidth costs 320 credits per GB ($4 at the list rate). Datacenter egress is free. This guide shows where Wayfern meters residential bytes and how to send fewer of them.

Where residential bytes are metered

Wayfern meters residential bytes in two places:

  • Browser sessions that use a residential exit: a country on the connection URL, or proxy=residential. Wayfern reads the bytes that the browser sends and receives every 5 seconds.
  • Web Data calls with proxy: "residential", and auto calls that Wayfern retries through a residential exit after a block.

These do not meter residential bytes:

  • The default datacenter exit.
  • Your own proxy on a browser link.
  • Search API calls. They bill per provider page only.
  • Web Data cache hits, because Wayfern fetches nothing.

The Usage page in the dashboard shows your residential GB. The credits field of a Web Data response shows only the endpoint charge. Residential bytes are a separate debit.

Start on the datacenter exit

On Web Data calls, keep proxy: "auto", the default. Only a blocked page goes to a residential exit. Use datacenter when you want a blocked error instead of a residential retry.

Browser sessions do not retry by themselves. Connect without a location first, and reconnect with a country only when the site blocks you:

import { chromium } from "playwright";

const base = `wss://api.wayfern.com/ws?token=${process.env.WAYFERN_LINK_TOKEN}`;
const target = "https://example.com/catalog";

async function open(extra = "") {
  const browser = await chromium.connectOverCDP(base + extra, { timeout: 120_000 });
  const page = await browser.contexts()[0].newPage();
  const response = await page.goto(target);
  return { browser, page, status: response?.status() ?? 0 };
}

let run = await open();
if ([401, 402, 403, 407].includes(run.status)) {
  await run.browser.close();
  run = await open("&country=US");
}
console.log(run.status, await run.page.title());
await run.browser.close();

These are the statuses that Wayfern’s own auto mode counts as a block. A browser with another exit is a different browser, so the second connection is a cold start.

Block what you do not need

On scrape, blockAssets is true by default. It blocks images, media and fonts. Wayfern ignores it when you ask for a screenshot or pdf output, and the Screenshot endpoint never blocks images.

Browser sessions block nothing by default. Block heavy resources in your own code:

const context = browser.contexts()[0];
await context.route("**/*", (route) => {
  const type = route.request().resourceType();
  if (type === "image" || type === "media" || type === "font") {
    return route.abort();
  }
  return route.fallback();
});

To remove ads and trackers as well, see Block ads and cookie banners.

Reuse what you already fetched

  • scrape takes maxAge in milliseconds. A cached capture up to that age comes back without a fetch, so it uses no bandwidth. The cache keeps pages for 1 hour, and pages over 2 MB are not cached. A cache hit is still a billed page, flagged "cached": true.
  • The cache key includes proxy, country, headers, mobile, blockAssets, waitFor and waitForSelector. Send the same values to get a hit.
  • extract does not bill source pages that come from the cache.
  • A crawl that visits a page again reuses its capture.
curl -X POST https://api.wayfern.com/api/v1/scrape \
  -H "Authorization: Bearer $WAYFERN_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/catalog",
    "proxy": "residential",
    "country": "us",
    "maxAge": 3600000
  }'

In a session, the browser’s own cache lasts only as long as the browser. Wayfern stops a browser when its last connection closes. Visit a batch of pages on one connection instead of one connection per page.

Cap the spend

If your balance cannot pay for residential bytes in a session, the socket closes with code 4408. To cap auto top-up, set a monthly spend limit. See Credits and spend limits and the usage rates.

Related guides