Rate Limiting and Abuse Prevention

Intermediate

Your API bill is also an attack surface

01 The Concept

AI inference is expensive per-call compared to typical web requests, which makes AI APIs an attractive target for two kinds of abuse: cost-based denial of service (flooding requests to run up your bill or exhaust a shared quota) and automated extraction or scraping (querying at high volume to reconstruct model behavior). Rate limiting has to account for both the request rate and the cost per request — a small number of very long, very expensive prompts can do as much damage as a flood of small ones.

Good abuse prevention layers per-user and per-IP rate limits, anomaly detection on usage patterns (sudden spikes, oddly systematic query sequences), and cost circuit-breakers that pause a client automatically rather than relying on someone noticing the bill after the fact.

02 Case Study

CASE"Model Denial of Service" (LLM04), OWASP Top 10 for LLM Applications

OWASP's working group named this category specifically because LLM inference breaks an assumption most rate-limiting design carries over from ordinary web APIs: that request count is a reasonable proxy for cost. Their write-up describes attacks built around this gap directly — a small number of deliberately long, context-heavy requests, or prompts engineered to make the model generate maximally long outputs, can consume as much compute and cost as a much larger flood of ordinary small requests, while sailing straight under a naive per-minute request cap.

→ Why it matters

The category exists because this gap was being actively exploited in real deployments before it had a name — which is exactly why rate limiting for an LLM API has to account for estimated cost per request (context length, expected output size), not just how many requests arrive per minute.

03 Key Points

01AI inference cost per call makes AI APIs a target for cost-based denial-of-service, not just traditional flooding

02Rate limits should account for cost per request, not just request count

03Anomaly detection on usage patterns catches abuse that stays under a naive rate limit

04Automated cost circuit-breakers respond faster than a human noticing the bill later

Next: Training Data Poisoning →