Handle API errors and retries

Every Wayfern API returns a stable error code that your code can branch on. This guide shows the error shape of each API, which errors to retry, how to back off and which failures are free.

The error envelope

The Web Data API, the Search API, the Captcha API and Operations return the same flat body:

{
  "error": "blocked",
  "code": 403,
  "request_id": "3b7c1d0e-9f2a-4c58-8d31-6e0a5b4c7d92",
  "message": "blocked: https://example.com/pricing",
  "reason": "https://example.com/pricing"
}

• error is a stable code. Branch on it, not on message.
• code is the HTTP status.
• request_id is also in the X-Request-ID response header. Send it when you report a problem.
• reason is the URL that failed on Web Data, the engine on Search, or the call class on Operations. On a 402, it is the billing reason. The Captcha API sends reason only on a 402.
• Operations add next, a short hint of what to try.

The account routes (tokens, billing, projects) use a different body: statusCode, message, and usually error with the name of the HTTP status.

What each status means

Status	error values	Retry?
400	validation, invalid_url	No. Fix the request. Unknown body fields are also a 400.
401	unauthorized	No. The token is missing, malformed, revoked, expired or lacks the scope.
402	billing_blocked	No. reason is insufficient_credits, spend_limit_reached or no_access.
403	blocked, forbidden	blocked: once, with proxy: "residential". forbidden: no.
404	not_found	No.
409	conflict, Captcha ambiguous_session	No. For ambiguous_session, send the full page.url() as websiteURL.
413, 415	too_large, unsupported_content	No.
422	unprocessable, Operations codes, Captcha codes	No. Change the input, or send a new captcha task.
429	rate_limited, Search captcha_detected	Yes, after a wait.
500	internal_error	Yes, with backoff.
503	upstream_error, navigation_failed, timeout, Search search_timeout, Operations drift and error, llm_unavailable, storage_unavailable, service_unavailable, Captcha timeout	Yes, with backoff. For a Web Data timeout, send a longer timeout. A captcha task that times out needs a new task.

Some codes have more detail:

• A 429 can come from three places. A route rate limit sends Retry-After in seconds. The per-account limit of 6 Web Data requests at once and the per-account limit of 5 open captcha tasks send no Retry-After. A site that rate-limits Wayfern also gives rate_limited. Wait a few seconds before you retry.
• blocked means the site refused the request. With proxy: "auto", Wayfern already tried a residential exit once.
• On Search, reason is the engine that failed. One engine can fail while others answer. The answer then lists it in meta.engines_failed.
• The Captcha API has its own 422 codes: invalid_site_key, invalid_domain, invalid_task and unsolvable. A 409 ambiguous_session means that two or more browser sessions match websiteURL and use different proxies. A 202 from the Captcha API is not an error: the task is not ready, so read it again with GET /api/captcha/solve/:id. See Solve captchas.

Operations call failures

A failed operation call has a reason that names its class:

reason	Status	Meaning
blocked	403	The site refused every tier the call could use.
rate	429	Too many requests to the site. Retry-After is set when the site gave one.
drift	503	The site changed. A repair runs in the background. Call again in a few minutes.
input	422	The examples still answer, so the site refused these arguments.
error	503	The site failed for another reason.

Learn refusals use codes such as login_wall, robots_denied, no_candidates and unsafe_url. Each comes with a next hint.

Back off and retry

Retry only 429 and 5xx. Use Retry-After when it is present. Otherwise wait longer after each try.

const RETRY = new Set([429, 500, 502, 503, 504]);

async function callWayfern(path, body, attempts = 4) {
  for (let attempt = 1; ; attempt++) {
    const res = await fetch(`https://api.wayfern.com${path}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.WAYFERN_API_TOKEN}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(180_000),
    });
    const data = await res.json();
    if (res.ok) return data;

    if (!RETRY.has(res.status) || attempt >= attempts) {
      throw new Error(`${res.status} ${data.error}: ${data.message} (request ${data.request_id})`);
    }
    const retryAfter = Number(res.headers.get("Retry-After"));
    const waitMs = retryAfter > 0 ? retryAfter * 1000 : 1000 * 2 ** attempt + Math.random() * 500;
    await new Promise((resolve) => setTimeout(resolve, waitMs));
  }
}

const page = await callWayfern("/api/v1/scrape", { url: "https://example.com", formats: ["markdown"] });
console.log(page.credits, page.data.metadata.title);

What a failure costs

• A Web Data request that fails costs nothing. Wayfern checks your balance before it opens a browser and debits only when it has output.
• A failed crawl page costs nothing. The pages before it stay billed.
• A captcha task that fails or times out costs nothing.
• An operation call costs 1 credit only when it answers with data or an empty result. A learn costs 3 credits only when it returns candidates.
• Search bills only the provider pages it returned.

A success that comes from the cache is still billed, except extract source pages. See Caching and freshness.

Browser sessions

The Browser API runs over a WebSocket, so it reports errors as close codes. 4005 means the session is not allowed, for example with too few credits. 4402 ends a running session when the balance or the spend limit can no longer pay. See Browser API for every code, and Web Data errors for the full sentinel table.