Defensive API Design: Rate Limiting, Idempotency, and Circuit Breakers

The Fallacy of Distributed System Reliability

In distributed architectures, transient network failures, noisy neighbors, and downstream latency spikes are unavoidable realities. Defensive API design ensures that a degraded downstream dependency does not collapse upstream gateways.

1. Distributed Idempotency via Unique Request Tokens

Payment and mutation endpoints must handle duplicate requests gracefully (e.g., when a mobile client retries an HTTP POST due to a dropped TCP connection):

async function handleCharge(request: Request, env: Env): Promise {
  const idempotencyKey = request.headers.get('Idempotency-Key');
  if (!idempotencyKey) {
    return new Response('Missing Idempotency-Key header', { status: 400 });
  }

  // Atomically claim key or fetch cached response from KV
  const cached = await env.KV.get('idemp:' + idempotencyKey);
  if (cached) {
    return new Response(cached, {
      headers: { 'Content-Type': 'application/json', 'X-Cache': 'HIT' }
    });
  }

  const result = await processPayment(await request.json());
  await env.KV.put('idemp:' + idempotencyKey, JSON.stringify(result), { expirationTtl: 86400 });
  return Response.json(result);
}

2. Circuit Breaker States

A circuit breaker tracks consecutive error percentages. When failures exceed a threshold (e.g., 50% over 10 seconds), the circuit Opens and fails fast immediately, shielding the downstream database from overload until a test probe confirms recovery.