How to use proxies with Wayfern

Pick a datacenter, residential or own-proxy exit for browser sessions, use auto or residential mode on Web Data calls, and see how residential bytes are billed.

Updated

Wayfern has two places where you choose an exit. Browser sessions choose it on the connection URL. Web Data calls choose it with the proxy field. This guide covers both and shows how residential bandwidth is billed.

Browser sessions

ExitHow you ask for itBandwidth
Default datacenterNo location params, or proxy=noneFree
Residentialcountry=US, or proxy=residential&country=US320 credits per GB
Your own proxyproxy=own, or proxyMode: "own" on the linkNot metered

Rules for the connection URL:

  • proxy= on the URL wins over the link’s default.
  • proxy=residential needs a country. Without it, the socket closes with 4021.
  • country, region, city, isp and session work only with a residential exit. With proxy=own or proxy=none, the socket closes with 4021.
  • On a link that uses its own proxy, add proxy=residential to get a residential exit.

Residential location and a sticky IP

wss://api.wayfern.com/ws?token=$WAYFERN_LINK_TOKEN&country=US&region=CA&session=run42

region, city and isp narrow the exit. The gateway checks each code against the live catalog. An unknown code closes the socket with 4007.

session takes letters and digits only. With it, Wayfern asks for the same exit IP for 24 hours, so a multi-step flow keeps one IP. Without it, the residential exit can change between the connections that the browser opens.

List the codes that you can use. These routes take a wf_ API token with any scope:

curl https://api.wayfern.com/api/proxy/locations/countries \
  -H "Authorization: Bearer $WAYFERN_API_TOKEN"
{
  "locations": [
    { "code": "US", "name": "United States" },
    { "code": "GB", "name": "United Kingdom" }
  ]
}

regions needs country. cities needs country and takes region. isps needs country and takes region and city.

Bring your own proxy

Save your proxy on the link. Link routes take a dashboard session token, not a wf_ API token. You can also do this on the link page in the dashboard.

curl -X PATCH https://api.wayfern.com/api/projects/$PROJECT_ID/links/$LINK_ID \
  -H "Authorization: Bearer $DASHBOARD_JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "proxyMode": "own",
    "ownProxy": {
      "scheme": "socks5",
      "host": "proxy.example.com",
      "port": 1080,
      "username": "user",
      "password": "secret"
    }
  }'
  • The scheme is http, https or socks5.
  • The proxy must resolve to a public address. Private, local and reserved ranges are refused.
  • Use a username and password, up to 255 bytes each. An IP allowlist does not work, because the address of the browser changes per session.
  • The gateway logs in to your proxy before it starts a browser, with an 8-second limit.
  • Wayfern encrypts the password at rest and never returns it. Responses show hasPassword.

POST /api/projects/{projectId}/links/{linkId}/proxy/test checks the saved proxy (owners and admins, 10 a minute). It returns ok, result, message and latencyMs. Close codes 4022 to 4026 tell you why a session could not use the proxy. See own proxy.

Web Data calls

Every Web Data endpoint takes proxy and country:

  • auto is the default. The first try uses a datacenter exit. If the site blocks it, Wayfern tries once more through a residential exit.
  • datacenter never uses a residential exit.
  • residential always uses one.

Wayfern counts a response as blocked when its status is 401, 402, 403 or 407, or when the page ends on a known challenge path. A 429 or a timeout is not retried.

country is a two-letter code. It applies only to a residential exit. Without it, the residential exit uses the default country of the deployment. Web Data residential exits rotate. They have no sticky session, and the Web Data API has no own-proxy option.

curl -X POST https://api.wayfern.com/api/v1/scrape \
  -H "Authorization: Bearer $WAYFERN_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/pricing",
    "proxy": "residential",
    "country": "de"
  }'

How residential bandwidth is billed

  • Residential bandwidth costs 320 credits per GB (10⁹ bytes), metered per byte, from the same balance as everything else. At the list rate, that is $4 a GB.
  • Wayfern counts the bytes that the browser sends and receives through the exit.
  • In a session, Wayfern reads the count every 5 seconds. If your balance cannot pay, the socket closes with 4408.
  • On Web Data calls, the bytes bill on top of the endpoint rate. This includes an auto retry. The credits field in the response shows only the endpoint charge.
  • The datacenter exit is free. Traffic through your own proxy is not metered as residential bandwidth. Session time bills as usual.

To spend less, read Reduce proxy bandwidth. The rates are on the pricing page.