Rate Limiters — How to build one?
Okay, so let’s start with the basics. What exactly is a rate limiter?
Let’s talk about a scenario where you build a service, more like an API that gives some kind information or data in response when a request is made to it. Your user base is getting wider and then one day you realise that you need to handle the number of requests a user can make to your API in a given amount of time so that it does not get exploited to the extent that there is a certain fraction of users who are making a huge number of calls to your API, while the server is not able to entertain the other fraction(as the server can handle a limited amount of requests at a time). This is where rate limiters come in the picture! All of the big tech giants use it for their public APIs and you can even find it on websites that are nicely build. Rate limiters put a bar on the number of requests that can be made to your API, or group of APIs in a certain interval of time by a particular user either on the basis of their IP address or usernames or some other unique identifier as per your wish.
Let’s start building a Rate Limiter
In this section we will discuss how to build a rate limiter. We will start with the basics, so that you can understand the core concept of it’s working.
Step-1: Building a basic NodeJS Server
We will start with building a simple NodeJS server that can simply accept requests coming from the client. For that you need to start with downloading some dependencies:
npm install express cors body-parser dotenv mysql2 nodemon momentOnce you have downloaded all these dependencies you are good to go. Build a file server.js that will server as our main server file to handle requests.
const express = require('express');
const app = express();
app.listen(8000, () => {
console.log("Listening on http://localhost:8000");
});You have built a basic server that will listen to all the requests incoming on port 8000. You can run the server by running the following command while being in the same directory as server.js
nodemon server.jsNow that we have build a basic server, we can start making our MySQL database and then connecting it to the server. It is suggested to build rate limiters using in-memory databases like redis, memcached, etc, as they are fast and have low latency. But for the sake of simplicity of this article we will be using a MySQL database.
Step-2: Connecting to a MySQL Database
Taking into consideration that you have a MySQL server up and running on MySQL Workbench and have the credentials, let’s see how you can connect your server with the MySQL database. Create a file database.js in the same directory as server.js and write the following code.
const mysql = require('mysql2');
// create a new mysql connection
const connection = mysql.createConnection({
host: 'localhost',
username: 'abcd',
password: 'abcde',
database: 'rateLimiter'
});
// connect to the database
connection.connect((error) => {
if (error) {
console.error('Error connecting to MySQL database:', error);
} else {
console.log('Connected to MySQL database!');
}
});
module.exports = connection;Step-3: Understanding Rate Limiting Algorithm
In order to build a rate limiter, there are various algorithms that you can use. Following are the most prominent/used ones:
- Token bucket
- Leaking bucket
- Fixed window counter
- Sliding window counter
- Sliding window log
The one that we will be using here is Fixed window counter algorithm. In a nutshell, the fixed window algorithm works in such a way that we have fixed intervals of window size along the time axis and we define the maximum number of requests that can be accepted in one time window. For example, say, 5 requests per minute would mean that our rate limiter will allow 5 requests in one minute for one user and then the user will be blocked to make any further request in that 1 minute window. As soon as the minute is completed, the user is free to make requests again! Here’s a representation of what we are dealing with. The user is allowed to make at most 3 requests per second:
As you can see in the graph above, any reuest that comes beyond the defined limit is not entertained by ther server. Although in real world application this is handled in a different way if the requests cannot be discarded. We generally make a queue of requests and keep on adding the incoming requests, once the time window moves to the next time frame, the unprocessed requests in the queue are handled, but you don’t need to worry about that if you don’t have a huge user base for now.
Step-4: Implementing the Rate Limiting Algorithm
Now that we know how rate limiting works, it’s time to actually implemnt it. Let’s make a file called “rateLimiter.js” in the same directory as our previous file “server.js” and “database.js” and write the following code:
const moment = require('moment');
const connection = require('./database');
const RATE_LIMIT_DURATION_IN_SECONDS = 60;
const NUMBER_OF_REQUEST_ALLOWED = 5;
module.exports = {
rateLimiter: async (req, res, next) => {
const userId = req.headers["user_id"];
const currentTime = moment().unix();
let isPresent;
connection.query('SELECT * FROM User WHERE user_id='+userId, (err, rows) => {
if(err){
console.log(err);
}else{
if(rows.length === 0){
// means that the current user is not present in the database
// so, we need to insert this user in the database
const newUserID = req.headers["user_id"];
const createdAt = currentTime;
const count = 1;
const data = [newUserID, createdAt, count];
connection.query('INSERT INTO User (user_id, createdAt, count) VALUES(?)', [data], (err, rows) => {
if(err){
console.log(err);
}else{
console.log("New user added to Database!", rows);
}
})
return next();
}else{
// means that the current user is present in the database
// fetch that user and check the conditions of rate limiting
connection.query('SELECT * FROM User WHERE user_id=' + userId, (err, rows) => {
if(err){
console.log("Error fetching user details");
}else{
const creationTime = rows[0].createdAt;
const prevCount = rows[0].count;
const difference = currentTime - creationTime;
if((difference < RATE_LIMIT_DURATION_IN_SECONDS) && (prevCount < NUMBER_OF_REQUEST_ALLOWED)){
// permission granted & update the count value in db
const updatedData = [userId, rows[0].createdAt, prevCount+1];
connection.query(`UPDATE User SET count=${prevCount+1} WHERE user_id=${userId}` , (err, rows) => {
if(err)
console.log(err);
else{
console.log("User updated!");
}
})
return next();
}else{
// check if difference > rate limit duration
if(difference > RATE_LIMIT_DURATION_IN_SECONDS){
const updatedData = [userId, creationTime, 1]
connection.query(`UPDATE User SET createdAt=${currentTime}, count=${1} WHERE user_id=${userId}`, (err, rows) => {
if(err){
console.log("User has entered a new time slot, but got some error!");
}else{
console.log("User updated: entered new time slot!");
}
})
return next();
}else{
console.log("Limit Reached!!!!")
return res.status(429).json({
"success": false,
"message": "Too many requests!"
})
}
}
}
})
}
}
})
}
}You can check out the code above to see how the code works. The basic flow is:
- Decide the size of time frame you want (I have used a window size of 60 seconds) and the number of requests allowed in 1 time frame (I have made it to be 5)
- Whenever a request comes through this middleware, the userId is stored in a variable and the time at which the request came is recorded.
- Now we check in our database if the user with this userId is present or not, if the user is not present, we simple push the userId and current time in our database
- If the user is present, we fetch the starting time of the user’s window frame and deduct it from the current time. If resulting time is greater than decided window size the user is allowed to make the request and the starting time of window in database is updated for the user. Although, if the resulting time is smaller than the decided window size, we fetch the number of requests that are stored in that time frame. If the number of requests is less than the decided maximum requests, we allow the user to make the request and simultaneously increment the counter of requests, while if this number is greater than the decided maximum requests, we send the status code of 429 in response to the user.
Step-5: Building path to handle incoming requests
Let’s get back to our server.js file and add this path and it’s callback function. We need to add the rateLimiter function as a middleware so that all the requests that come to the server are first processed by the rateLimiter function before they hit any path.
const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser');
const connection = require('./database.');
const rateLimiter = require('./rateLimiter')
const app = express();
app.use(cors({
origin: '*'
}));
app.use(bodyParser.json());
app.use(rateLimiter.rateLimiter);
app.get('/ping', async (req, res) => {
res.status(200).json({
"success": true,
"message": "Request validated!"
})
});
app.listen(8000, () => {
console.log("Listening on http://localhost:8000");
});Step-6: Testing our Rate Limiter
You can use any method to make a request to your server for path ‘/ping’ and check if your rateLimiter method is working as expected. Let’s see how it looks on Postman.
What’s Next?
Well, now you know how to build a rate limiter. It is a small, but a very essential feature that you can incorporate in your server if you are building APIs for which you need streamlining of users. So go ahead and integrate this in your website, API, microservice, and a whole wide range of applications of Rate Limiters.
If you want to optimise it further, you can do the following:
- Use the Sliding window counter algorithm as it will provide an even better basis to limit the number of requests by users as compared to other algorithms.
- Use an in-memory database like Amazon dynamo, Redis, Memcached, etc at the place Relational databases like MySQL or PostgreSQL as they offer better speed because of the way data is stored in them.
You can check out the entire code for this here.
Feel free to get in touch with me on LinkedIn, or follow me on Twitter. 🐘
