Detailed design

• Rules are stored on the disk. Workers frequently pull rules from the disk and store them in the cache.

• When a client sends a request to the server, the request is sent to the rate limiter middleware first.

• Rate limiter middleware loads rules from the cache. It fetches counters and last request timestamp from Redis cache. Based on the response, the rate limiter decides:

  • if the request is not rate limited, it is forwarded to API servers.

  • if the request is rate limited, the rate limiter returns 429 too many requests error to the client. In the meantime, the request is either dropped or forwarded to the queue.