Skip to content
←Resilience & Messaging

// Resilience & Messaging · 08

Rate Limiting

Too many requestsleads toExcess gets a 429

Cap how many requests a caller gets per window of time.

How it fits together

Rate Limiting structure«client»Caller A«client»Caller BToken bucket100 tokens max+20 tokens per sHTTP 429Search servicerequestrequesttoken takenbucket emptyRetry-After
uses / callsfails / error path
  • Caller A uses / calls Token bucket (request)
  • Caller B uses / calls Token bucket (request)
  • Token bucket uses / calls Search service (token taken)
  • Token bucket fails / error path HTTP 429 (bucket empty)
  • HTTP 429 fails / error path Caller B (Retry-After)

Spot it

  • One tenant slows the API for everyone
  • A buggy client or scraper floods an endpoint
  • A supplier throttles you for exceeding quota

It costs

Legitimate bursts get rejected, and a truly global limit needs shared state.

In code

builder.Services.AddRateLimiter(limits =>{    limits.RejectionStatusCode = StatusCodes.Status429TooManyRequests;    limits.OnRejected = (context, ct) =>    {        if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))        {            context.HttpContext.Response.Headers["Retry-After"] =                ((int)retryAfter.TotalSeconds).ToString();        }​        return ValueTask.CompletedTask;    };    limits.AddPolicy("search", context =>        RateLimitPartition.GetTokenBucketLimiter(            partitionKey: context.Request.Headers["X-Api-Key"].ToString(),            factory: _ => new TokenBucketRateLimiterOptions            {                TokenLimit = 100,                TokensPerPeriod = 20,                ReplenishmentPeriod = TimeSpan.FromSeconds(1),                QueueLimit = 0,                AutoReplenishment = true,            }));});
Go deeperThe full explanation and what to ask for · 1 min readExpand

Fixed window is simple but allows a double burst at the boundary; sliding window smooths it. Token bucket allows short bursts up to the bucket size while holding the average rate. A concurrency limiter caps requests in flight rather than per second.

The classic mistake is an in-memory limiter behind a load balancer: with 4 instances, a limit of 100 per second is really up to 400. Divide the limit per instance, or keep counters in a shared store, such as a cache like Redis, and pay the extra round trip.

Server-side limits protect you; client-side limits keep you inside someone else's quota. Callers that get a 429 must honour Retry-After and back off, or the limiter just turns load into rejected load.

Ask for it

  • token bucket
  • sliding window
  • 429 Too Many Requests
  • Retry-After
  • partition key
  • quota

Add rate limiting to /api/search: a token bucket partitioned by API key, 100 tokens refilling 20 per second, no queue, rejecting with 429 and Retry-After. Anonymous callers get a fixed window of 30 per minute per IP. Note the limits are per instance.

Connects to