High-level architecture

The basic idea of rate limiting algorithms is simple. At the high-level, we need a counter to keep track of how many requests are sent from the same user, IP address, etc. If the counter is larger than the limit, the request is disallowed. Where shall we store counters? Using the database is not a good idea due to slowness of disk access. In-memory cache is chosen because it is fast and supports time-based expiration

strategy. For instance, Redis is a popular option to implement rate limiting. It is an in-memory store that offers two commands: INCR and EXPIRE.

• INCR: It increases the stored counter by 1.

• EXPIRE: It sets a timeout for the counter. If the timeout expires, the counter is automatically deleted.

• The client sends a request to rate-limiting middleware.

• Rate-limiting middleware fetches the counter from the corresponding bucket in Redis and checks if the limit is reached or not.

  • If the limit is reached, the request is rejected.

  • If the limit is not reached, the request is sent to API servers. Meanwhile, the system increments the counter and saves it back to Redis.