The In-App AI Assistant That Does Not Stall When a Key Hits Its Limit

By Theodora Christou · Co-Founder, Dabster Labs · Client engagement, anonymised at the client’s preference

A video ad agency runs a lot of its day through the AI assistant built into its CRM: asking about leads, drafting emails, creating follow-ups. The problem was simple and painful. The assistant ran on a single AI key, so the moment that key hit a rate limit or ran out of quota, the assistant stopped mid-task with nothing to fall back on. Now the assistant keeps working straight through a limit, and the person using it usually never notices.

The situation

One key, one point of failure. When that key tripped a rate limit, every request through the assistant failed until the limit reset, which could be minutes or longer. There was no second key to try and no automatic recovery. For a team that leans on the assistant during a busy day, a dead assistant at the wrong moment is a real interruption.

What we built

We gave the owner a place to configure several AI keys and providers in a priority order. When the active key hits a rate limit, the request transparently fails over to the next key in the list. The tripped key goes on a short cooldown, and once its limit resets the assistant reverts to the preferred key on its own, with no manual step. An optional base URL lets a connection point at any OpenAI-compatible provider, so keys from OpenAI, Groq, OpenRouter, Grok, Mistral, Gemini, DeepSeek or a self-hosted model can all sit in the same chain. That means a free backup key can quietly cover a paid primary.

What changed

• The assistant keeps working through a rate limit instead of stopping mid-task.
• The owner can list several keys and providers in priority order, and a free backup key can cover a paid primary.
• Failover and recovery both happen automatically, with no manual switch to flip. Admins get an in-app notice when it fails over and again when it recovers.
• A single-key setup behaves exactly as before, so nothing changed for anyone who wants the simple path.
• All keys are encrypted at rest and are never returned by the API or written to the logs.

How it works

Keys are tried in the order the owner sets. When a request comes back as rate limited, the assistant moves to the next key and completes the task on it. The key that tripped is parked on a short cooldown so it is skipped while its limit is still in effect, then brought back into rotation once that window passes, at which point the preferred key takes over again. The optional base URL is what lets one connection talk to any OpenAI-compatible endpoint, so mixing providers needs no extra code. Keys and base URLs are encrypted at rest, kept out of the environment and never exposed through the API or logs.

Where it stands

The failover is live in the CRM. Because it touches keys and encrypted secrets, we hardened it over three adversarial review passes before it shipped. A single-key configuration is untouched, and multi-key setups get the resilience for free. We have not put a number on stalls avoided yet (measuring), but the day-to-day effect is that the assistant stays up when a key does not.

Want an in-app assistant that keeps working when a key runs out? Talk to us about your build.

Related service: AI Chatbots