Skip to content
←Resilience & Messaging

// Resilience & Messaging · 04

Retry with Backoff

Transient failureleads toSpaced, jittered retries

Try again, waiting longer each time, with a little randomness.

How it fits together

Retry with Backoff structure«client»Inventory clientHTTP client10s total budgetAttempt 1503 / timeoutWait 200ms ±jitterWait 400ms ±jitterAttempt 2503 / timeoutAttempt 3 OK1 callfailsretryfails againretry
uses / callsfails / error path· numbered steps replay in order
  • Inventory client uses / calls Attempt 1 (1 call)
  • Attempt 1 fails / error path Wait 200ms ±jitter (fails)
  • Wait 200ms ±jitter uses / calls Attempt 2 (retry)
  • Attempt 2 fails / error path Wait 400ms ±jitter (fails again)
  • Wait 400ms ±jitter uses / calls Attempt 3 OK (retry)

Spot it

  • Brief blips surface as user-facing errors
  • Retries from every client land in lockstep
  • A recovering service gets flattened again

It costs

Retries add latency and load, and they multiply when every layer retries.

In code

services.AddHttpClient<InventoryClient>()    .AddResilienceHandler("inventory", pipeline =>    {        pipeline.AddRetry(new HttpRetryStrategyOptions        {            MaxRetryAttempts = 3,            BackoffType = DelayBackoffType.Exponential,            Delay = TimeSpan.FromMilliseconds(200),            UseJitter = true,            ShouldHandle = args => ValueTask.FromResult(                args.Outcome.Exception is not null                || (int?)args.Outcome.Result?.StatusCode is 408 or 429 or >= 500),        });​        // Added after the retry, so the timeout applies to each attempt.        pipeline.AddTimeout(TimeSpan.FromSeconds(2));    });
Go deeperThe full explanation and what to ask for · 1 min readExpand

Retry only transient faults: timeouts, 408, 429, 503, connection resets. A 400 or a validation error fails the same way every time, and retrying a non-idempotent call can do the work twice.

The classic mistake is retries without jitter. Clients that failed together back off together and return together, a thundering herd that knocks the dependency over just as it recovers. Randomising each delay spreads them out.

Retries multiply across layers: 3 retries at each of 3 hops is up to 64 calls at the bottom for one user request. Retry at one layer, cap the total with an outer timeout, and pair it with a circuit breaker so a dead dependency stops being retried at all.

Ask for it

  • transient fault
  • exponential backoff
  • jitter
  • Retry-After
  • per-attempt timeout
  • retry budget

Add a retry policy to the Inventory HTTP client: retry only 408, 429, 5xx and timeouts, max 3 retries, exponential backoff from 200ms with jitter, honour Retry-After, 2s per-attempt timeout inside a 10s total. Retry only GETs and requests with an idempotency key.

Connects to