Slow, empty and failed responses

A brand context call can be slow the first time that someone asks for a domain. The call can be empty if the vendor cannot read the site. The vendor can also refuse the call because of a rate limit or a credit limit. Each vendor signals these cases in a different way. If the vendor tells you to wait, wait. If it has nothing to return, use your fallback. If you have no credits left, stop.

Endpoint	Still working	Nothing to return	Slow down	Out of credits
Brandfetch Brand Context API	503 with x-bf-error: generation_in_progress and Retry-After	404, 422 if the site blocks bots, or 204 with cachedOnly	429 (quota)	429
Brandfetch Brand API	404 crawl_queued	404, or 204 with cachedOnly	Not listed	429
Context.dev	The request stays open. timeoutOpts can return partial results	400 NOT_FOUND	429 with Retry-After	402 USAGE_EXCEEDED
Firecrawl	408 if the page is too slow	A page's own 404 comes back as a document	429 with Retry-After	402
Logo.dev	202 with Retry-After: 30. Stop after 5 minutes	404 not_found	429. Wait for RateLimit-Reset	402
Rainbrand Brief	status: pending or processing	404 BRAND_NOT_FOUND, or 409 if the brand is not ready	429 with Retry-After	402 INSUFFICIENT_CREDITS

One retry loop for all of them

const WAIT = new Set([202, 429, 503]);

export async function fetchWithWaits(url: string, init: RequestInit, giveUpAfterMs = 300_000) {
  const started = Date.now();
  for (;;) {
    const res = await fetch(url, init);
    if (!WAIT.has(res.status)) return res;
    // A 503 from Brandfetch's payment service also carries Retry-After.
    const seconds = Number(res.headers.get("retry-after") ?? res.headers.get("ratelimit-reset") ?? 30);
    if (Date.now() - started + seconds * 1000 > giveUpAfterMs) return res;
    await new Promise((r) => setTimeout(r, seconds * 1000));
  }
}

export function isEmpty(res: Response): boolean {
  return [204, 404, 422].includes(res.status);
}

import time
import httpx

WAIT = {202, 429, 503}

def fetch_with_waits(method: str, url: str, give_up_after: float = 300, **kwargs) -> httpx.Response:
    started = time.monotonic()
    while True:
        res = httpx.request(method, url, timeout=60, **kwargs)
        if res.status_code not in WAIT:
            return res
        seconds = float(res.headers.get("retry-after") or res.headers.get("ratelimit-reset") or 30)
        if time.monotonic() - started + seconds > give_up_after:
            return res
        time.sleep(seconds)

Context.dev answers "not found" with a 400 and the code NOT_FOUND. Before you treat a 400 as a bug, read the code in the body.

Waiting for a Rainbrand Brief

Rainbrand prepares a Brief after you request it. Get a Brief with wait=true holds the connection until the Brief is done or the timeout passes (up to 295 seconds). For this reason, the loop is short:

async function waitForBrief(id: string) {
  for (;;) {
    const res = await fetch(`https://www.rainbrand.com/api/v1/briefs/${id}?wait=true&timeout=120`, {
      headers: { "x-api-key": process.env.RAINBRAND_API_KEY! },
    });
    const { data } = await res.json();
    if (data.status === "completed") return data;
    if (data.status === "failed") throw new Error(`${data.error.code}: ${data.error.message}`);
  }
}

Our Brief took 42 seconds. In a failed Brief, data.error.retryable tells you if you can request the Brief again.

What to do when there is nothing

Do not send the model an empty brand. Tell the model what you do not have, for example: "No brand context was available for this company. Write in a neutral, plain voice. Do not describe its products." Then show the person what was missing, so that they can add it.