How to Handle API Rate Limits Without Losing Work
How should you handle API rate limits?
To handle API rate limits, keep unfinished work in a durable queue, follow the provider's retry instructions and reduce the pace of requests. Retry only when repeating the operation is safe, with a waiting period and a clear stopping point. Show users what is waiting, what has finished and whether they need to act.
The product decision comes first: how long can this work wait before it stops being useful? A background catalogue update and an action someone needs immediately should not share the same recovery policy.
What does a rate limit mean for your integration?
A provider sets boundaries on how much work your integration can request. Check its documentation for what counts towards those boundaries and whether they apply to an account, a particular operation or another shared allowance.
A rate-limit response may use HTTP status 429 and may include a Retry-After header.
Treat the response as a reason to control demand. Do not assume every failed request is a rate limit: an expired connection, invalid input and a temporary service failure need different handling.
Before approving an integration, ask for a written policy covering its limits, how delayed requests resume and which operations require special care before retrying.
How should a queue preserve unfinished work?
Design the queue so accepted work survives a restart. Each job should record its intended action, current state, next eligible attempt and enough information to recognise the same logical operation again.
Use explicit states such as waiting, processing, completed, failed and cancelled. Define what happens if processing stops before completion is recorded, including how the next worker checks whether the outside action already happened.
Set boundaries around the queue:
- Limit how much pending work each customer can create.
- Give time-sensitive actions priority over bulk updates.
- Preserve order where a later action depends on an earlier one.
- Expire work that has become irrelevant.
- Decide whether a newer update can replace an older pending update.
Keep separate customers moving where their allowances are independent. Where several workers share an allowance, make them share the same request budget.
These choices belong in the wider design of scalable and robust architectures, alongside capacity and recovery planning.
Which retry instructions should take priority?
Follow the provider's documented retry instructions before applying your own fallback.
If it supplies a recognised waiting instruction, schedule the next attempt no earlier than that instruction allows. If the instruction is absent or unusable, apply a conservative fallback and record why it was needed.
Put this decision in one place. Avoid having the connection library, job runner and application each retry the same operation independently.
Specify what the integration does when:
- The provider gives no waiting instruction.
- Its response conflicts with its published documentation.
- A single operation has a different limit from other requests.
- Repeated limiting continues beyond the job's useful lifetime.
Do not promise a completion time merely because the next attempt has been scheduled. That attempt still needs to succeed.
How do backoff and jitter help recovery?
Use backoff to increase the waiting period after repeated retryable failures. Add a small random variation, called jitter, so queued jobs are scheduled at different times.
For an illustrative example, a fallback policy might wait roughly 2 seconds, then 4, then 8, with variation around each interval. These are example values, not provider recommendations. A provider instruction requiring a longer wait should take priority.
Cap both the waiting interval and the total retry effort. Decide what happens when the attempt limit or job deadline is reached: stop, request review or offer a deliberate restart.
Control the recovery pace too. Releasing every waiting job as soon as a pause ends defeats the purpose of a shared request budget. Resume within that budget and adjust cautiously as requests succeed.
When is repeating a request safe?
Make retry safety an explicit property of each operation. A request to fetch information and a request to create a charge deserve separate decisions.
For actions with side effects, check whether the provider offers an idempotency mechanism: a documented way to recognise repeated attempts as the same operation. Its scope and retention period are provider-specific.
Where supported, keep the same operation identifier across retries. Generating a fresh identifier for each attempt would undermine that design.
If the outcome is uncertain, check the provider's records before repeating the action. A timeout should leave the job in an uncertain state until you can establish what happened.
| Situation | Recovery decision to define |
|---|---|
| Request explicitly rejected before processing | Reschedule if the provider permits it |
| Read request temporarily limited | Retry within its deadline and request budget |
| Write request with documented duplicate protection | Retry according to that protection's rules |
| Write request with an unknown outcome | Reconcile before repeating |
| Invalid input or lost access | Pause and request the required correction |
What should users see while work is delayed?
Separate saving the user's request from completing the outside action. Use “Saved and waiting to sync” only after the request has actually been saved. Reserve “Completed” for a confirmed result.
A useful status message explains what is happening and whether the user needs to act:
“Your export is waiting for the connected service. We will retry automatically. You can leave this page.”
Show completed and pending items separately when a batch is partly finished. Display when information was last updated if users might otherwise mistake an older result for a current one.
Offer cancellation where the action can still be stopped. Make manual retry reuse the existing job rather than silently creating another operation. When automatic recovery stops, give a specific next step instead of leaving the status at “Processing”.
What should you monitor?
Build monitoring around whether useful work is getting completed, with request counts as supporting evidence.
Track pending jobs, the age of the oldest job, completed work, rate-limit responses, retry attempts and jobs that stop permanently. Where the provider exposes usage information, compare it with your own records.
Break these views down by customer and operation where appropriate. Set alerts around the product's delay commitments and the queue's capacity.
Use this acceptance checklist:
- Can we identify whose work is delayed?
- Can we explain why it is waiting?
- Can we see whether the backlog is shrinking?
- Can we detect jobs that are no longer being attempted?
- Can we investigate without logging credentials or unnecessary personal data?
Monitoring access and retained data also deserve the controls discussed in data security in modern applications.
What common mistakes should you avoid?
Retrying everything. Define retryable conditions. Invalid input needs correction; waiting does not correct it.
Keeping the only copy in memory. Require pending work to survive a restart before calling it accepted.
Retrying at several layers. Assign one component responsibility for scheduling another attempt.
Ignoring shared limits. Coordinate workers that consume the same allowance.
Keeping obsolete jobs forever. Set deadlines and rules for replacing stale updates.
Hiding uncertainty. Show an unresolved outcome honestly, especially for actions with financial or other lasting effects.
Testing only eventual success. Test repeated limiting, missing instructions, restarts, partial completion and uncertain write outcomes. These scenarios fit naturally into software testing and quality assurance.
What questions do product owners ask?
Should every rate-limited request be queued?
No. Queue work that remains useful after a delay. For an immediate interaction, define a short waiting budget and a clear response when it cannot finish within that budget.
Should users reconnect their account?
Ask them to reconnect only when the connection requires renewed access. Keep that action separate from a temporary request limit.
Will a higher allowance solve the problem?
Evaluate it when legitimate demand exceeds the available allowance. Keep queueing, retry safety and status handling in the design so recovery still has clear rules.
What proves the integration is ready?
Demonstrate that accepted jobs survive interruption, retries respect the configured limits, uncertain actions are reconciled and users can distinguish waiting from completion. Agree on the maximum useful delay before accepting the integration.