# Retries, timeouts & cancellation

> Recover from temporary failures without duplicating uncontrolled work.

## Set a request deadline

Give every Router call an application-level timeout. Choose a deadline that fits the model and prompt, then show a clear waiting state. A network connection staying open does not guarantee that generation will finish.

## Retry only temporary failures

Good retry candidates include `429` responses and selected `503` responses such as `no_priced_provider_available` or `request_capacity`. Respect `Retry-After` when present. `no_provider_for_length` is not worth an immediate retry: shorten the request, or wait for its 60-second `Retry-After`. Use exponential backoff, random jitter, and a small attempt limit.

```typescript
const retryable = new Set([429, 503]);

async function waitBeforeRetry(response: Response, attempt: number) {
  const header = response.headers.get("retry-after");
  const headerSeconds = header !== null && /^\d+$/.test(header)
    ? Number(header)
    : NaN;
  const seconds = Number.isSafeInteger(headerSeconds)
    ? headerSeconds
    : Math.min(2 ** attempt, 8);
  const jitter = Math.random(); // Add at most one second after Retry-After.
  const delayMs = (seconds + jitter) * 1000;
  if (delayMs > 2_147_483_647) {
    throw new Error("Retry-After exceeds the supported wait; stop automatic retries");
  }
  await new Promise((resolve) => setTimeout(resolve, delayMs));
}
```

Do not retry malformed requests, unsupported parameters, invalid credentials, or an unknown model until you change the request.

## Reuse the request ID for a retry

Create one UUID for each logical request and send it as `Idempotency-Key`. If you lose the response, check `GET /v1/credits/requests/{request_id}` and retry only with the **same** UUID and unchanged body. A new UUID authorizes another paid request and can create another charge. Keep automatic retries bounded.

A same-UUID retry never starts a second paid run. Each case below shows what Router returns.

| Retry | Result |
| --- | --- |
| Same body, headers, and credential; not streaming | `Saved answer`. Returns the saved answer once the original request settles. Otherwise settlement_pending (503) asks you to check the receipt. |
| Original request still being admitted | `409 request_in_progress`. Router waits briefly, then returns request_in_progress. Check the receipt before retrying. |
| Changed body, credential, or X-Max-Charge-Credits | `409 idempotency_conflict`. The UUID already belongs to different terms. Resend the original terms, or use a new UUID. |
| Streaming request | `409 request_already_exists`. Streams are not replayed. Read the receipt instead. |
| A different Idempotency-Key | `A separate paid request`. A new UUID authorizes another paid request and can create another charge. |

## Recover a selected offer

A manual selection never silently falls back. For `provider_price_changed`, fetch the current offer and review its new price. For `provider_offer_unavailable` or `selected_provider_unavailable`, wait where appropriate, choose another eligible offer, or explicitly choose Auto. A `self_provider_only` result requires another account's provider.

Changing the offer, pricing revision, or routing choice changes the request body and requires a new UUID. If the previous outcome is uncertain, check its receipt first. Keep the same UUID and unchanged body when recovering the original request. See [choosing a provider](https://router.optimai.network/docs/choosing-a-provider.md).

## Scale deadlines with prompt size

The examples use short deadlines suited to short prompts. Long prompts take much longer before the first word: a provider may need several minutes to read a prompt near a 32K context on modest hardware. Raise your client deadline for long inputs and stream them (a non-streaming request above about 24 KB of message text is rejected with `stream_required`), and keep the deadline above the time you are willing to wait rather than retrying a request that is still running.

## Stop receiving when the user leaves

Abort the HTTP request if the user stops receiving, navigates away, or exceeds your deadline. This stops your local response handling; it does not guarantee cancellation of accepted paid work. Generation and settlement can continue after disconnect, and completed work can incur its normal charge within the accepted request maximum. Accepted provider-reported partial usage can also be charged; token counts are not independently verified. Keep the request UUID and check its receipt before starting another paid request.

```typescript
import { randomUUID } from "node:crypto";

const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 45_000);
const requestId = randomUUID(); // Keep for receipt lookup and same-request retry.

try {
  const response = await fetch(`${process.env.ROUTER_BASE_URL}/chat/completions`, {
    method: "POST",
    signal: controller.signal,
    headers: {
      Authorization: `Bearer ${process.env.ROUTER_API_KEY}`,
      "Content-Type": "application/json",
      "Idempotency-Key": requestId,
      "X-Max-Charge-Credits": "5.000000",
    },
    body: JSON.stringify({
      model: "gemma3:1b",
      messages: [{ role: "user", content: "Hello!" }],
    }),
  });
  // Inspect response.ok and the error body before reading a success payload.
} finally {
  clearTimeout(timeout);
}
```

> **Never join partial generations**
> If a stream ends with an error after returning text, keep that text marked incomplete. Do not append the output of a retry and present both attempts as one model answer.

Use the caller UUID and matching `X-Request-Id` response header to correlate an attempt with [usage history](https://router.optimai.network/docs/usage-and-observability.md) and its credit receipt.