API servers are everywhere—anytime you connect to or refresh a web application, your browser is sending out requests for the latest data. We’ll take a look at how that’s handled behind the scenes, and how you can code your own API server.
What Is an API Server, and How Does It Work?
Here’s an example. You have a web application that needs to connect to a database and fetch some data, like posts made by users on a social media site. You obviously can’t have the users connect directly to your database server, as that’s a huge security flaw leaving connection details in your app.
The solution is to put a server in the middle that takes requests from the user, interprets them, then securely queries the database for the requested information. The user never has to see the database; in fact, it’s often ran in a private subnet to make it inaccessible from anything except your API servers.
This allows clients to access your data securely over HTTP/HTTPS. Because the API server is owned and controlled by you, that connection can be secured more easily. Though you’ll still need some form of authentication to verify that users are signed in as who they say they are.
The API server isn’t meant to be doing the heavy lifting on your website. For most static content, you’ll want to use a traditional web server like NGINX, which will be much more performant. For example, you’ll serve index.html and bundle.js using your NGINX server, which the client will unpack and run. The client side JavaScript application will then make a request to the API server, which will handle it from there.
You can create API servers with any language—all you need is the ability to listen to and respond to HTTP requests, and the ability to connect to a database, both of which have many libraries for any language. In this guide, we’ll be using JavaScript with Node.JS, using Express for HTTP and route handling. Another popular option using C# is ASP.NET from Microsoft.
If you’re on AWS, you might want to look into AWS’s API GateWay. Rather than running a full API server, API GateWay provides a cheap managed solution that can act as a frontend for Lambda functions and RDS.
Setting Up an API Server with Express.JS
Express.JS is a web application framework that runs on Node (server-side JavaScript). It’s very simple to use, and pairs well with client-side JavaScript applications.
Make a new folder for the server, and run npm init to set up your package.json. Create App.js, and paste the following in:
const express = require(‘express’)
const app = express()
const port = 3000
app.get(‘/’, (req, res) => res.send(‘Message from Express route handler: Hello World!’))
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
This is the basic boilerplate for a simple Express server. The server itself is created with express(). After that, routes and their associated handler functions are registered. In this case, app.get(‘/’, handler) registers a handler function for GET requests on the / route. There are other handler functions for the other HTTP verbs, like POST and PUT.
In the handler, the req object represents the HTTP request, with res representing the response. Then res.send will send basic data back, like a string or JSON object. You can also use Express as a traditional web server with res.sendFile, but we recommend using a native one for performance with static object.
To start, you can run node App.js, but nodemon is better as it will monitor the base file for changes and restart automatically.
npm i -g nodemon
nodemon app.js
If you visit localhost:3000 in your browser, you should see express responding to requests.
In addition to the multiple handler functions, you can also match many different routes. Explicit routing is easy—app.get(‘/users’) will handle requests made to /users. You can also match routes using wildcards and regular expressions. You can even nest router handlers; for example, creating a new handler specifically for a subdirectory like ‘/users’, which is seperate from the main app and usually stored and exported from its own file.
This is all still being served over HTTP though, so the first upgrade will be to set up HTTPS. This is easy to handle; all you’ll need is your SSL private key and certificate. If you’re just testing, you can generate a self signed cert using openssl:
openssl req -nodes -new -x509 -keyout server.key -out server.cert
You’ll have to manually trust this cert when Chrome freaks out, but it will work. Rather than calling listen on app, we’ll set up the HTTPS server, pass it the app, then listen on that.
const express = require(‘express’)
var https = require(‘https’)
var fs = require(‘fs’)
const app = express()
const port = 3000
app.get(‘/’, (req, res) => res.send(‘Message from Express route handler: Hello World!’))
https.createServer({
key: fs.readFileSync(‘server.key’),
cert: fs.readFileSync(‘server.cert’)
}, app)
.listen(port, () => console.log(`Example app listening on port ${port}!`))
Hooking Up a Database
This step is dependant on your specific database, but really, it’s nothing special. All you’ll need to do is install the Node.JS driver for your database, give connection details to Express, and connect to it like you would from normal JavaScript (or whatever other language you’re using to build your API server).
For this guide, we’ll use MongoDB, a JSON document database that is often paired with Express. If you have Docker, you can run it in a container for testing:
docker run -d -p 27017-27019:27017-27019 –name mongodb mongo:4.0.4
You can use MongoDB Compass to connect to it and perform administrative tasks.
On the Express side of things, you’ll need to install the MongoDB NPM package to connect to it.
npm install mongodb
Because we don’t want Express to start until after the database is connected, we’ll move app.listen() inside of MongoClient.connect().
const express = require(‘express’)
var https = require(‘https’)
var fs = require(‘fs’)
const MongoClient = require(‘mongodb’).MongoClient
const app = express()
const port = 3000
app.get(‘/’, (req, res) => res.send(‘Message from Express route handler: Hello World!’))
var db;
MongoClient.connect(‘mongodb://localhost:27017/’, (err, client) => {
if (err) return console.log(err)
db = client.db(‘test’) // whatever your database name is
https.createServer({
key: fs.readFileSync(‘server.key’),
cert: fs.readFileSync(‘server.cert’)
}, app)
.listen(port, () => console.log(`Example app listening on port ${port}!`))
})
Now, you can connect to Mongo using the db variable and any standard methods. You’ll want to make sure to use async / await though, or you’ll be returning unhandled promises.
app.get(‘/’, async (req, res) => {
const response = await db.collection(‘test’).find().toArray();
res.send(response)
})
This connects to the database, returns the contents of the test collection, and spits it out as a JSON array.
Because Express is just the frontend for these functions, you don’t have to define all of your logic in this one file—you can split it up into modules, and import what you need to call from the Express route handlers.
Caching API Requests for Performance
Express isn’t the fastest thing around. If you’re planning on using it in production, you’ll likely want to implement some form of caching so that it doesn’t have to doesn’t have to do expensive lookups on repeatable requests.
One thing to note is that you should never cache anything that requires authentication to view, like a user’s personal information. The general rule is that if the page is the same for everyone, you can cache it for everyone. PUT, DELETE, and POST methods should never be cached.
You can use in-memory store like Redis for this, but a simple memory cache works fairly well. Install memory-cache from npm:
npm i memory-cache
Then, in App.js, create a new caching middleware before your routes:
const cache = require(‘memory-cache’);
let memCache = new cache.Cache();
let cacheMiddleware = (duration) => {
return (req, res, next) => {
let key = ‘__express__’ + req.originalUrl || req.url;
let cacheContent = memCache.get(key);
if(cacheContent){
res.send( cacheContent );
return;
} else {
res.sendResponse = res.send;
res.send = (body) => {
memCache.put(key,body,duration*1000);
res.sendResponse(body);
}
next()
}
}
}
Then, on the routes that you are able to cache, you can use this middleware before the primary handler, to cache results for X seconds:
app.get(‘/products’, cacheMiddleware(30), function(req, res){
…
});