Step 2 - Propose high-level design and get buy-in

Where to put the rate limiter?

Intuitively, you can implement a rate limiter on either the client or server side.

• Client-side implementation. Generally speaking, a client is an unreliable place to enforce rate limiting because client requests can easily be forged by malicious actors. Moreover, we might not have control over the client implementation.

• Server-side implementation.

Besides the client and server-side implementations, there is an alternative way. Instead of putting a rate limiter at the API servers, we create a rate limiter middleware, which throttles requests to your APIs

Assume our API allows 2 requests per second, and a client sends 3 requests to the server within a second. The first two requests are routed to API servers. However, the rate limiter middleware throttles the third request and returns an HTTP status code 429. The HTTP 429 response status code indicates a user has sent too many requests.

Cloud microservices have become widely popular and rate limiting is usually implemented within a component called API gateway. API gateway is a fully managed service that supports rate limiting, SSL termination, authentication, IP whitelisting, servicing static content, etc. For now, we only need to know that the API gateway is a middleware that supports rate limiting.

While designing a rate limiter, an important question to ask ourselves is: where should the rater limiter be implemented, on the server side or in a gateway? There is no absolute answer. It depends on your company’s current technology stack, engineering resources, priorities, goals, etc. Here are a few general guidelines:

• Evaluate your current technology stack, such as programming language, cache service, etc. Make sure your current programming language is efficient to implement rate limiting on the server side.

• Identify the rate-limiting algorithm that fits your business needs. When you implement everything on the server side, you have full control of the algorithm. However, your choice might be limited if you use a third-party gateway.

• If you have already used microservice architecture and included an API gateway in the design to perform authentication, IP whitelisting, etc., you may add a rate limiter to the API gateway.

• Building your own rate-limiting service takes time. If you do not have enough engineering resources to implement a rate limiter, a commercial API gateway is a better option.