Node.js and Redis: Caching That Works

Performance matters in a web app. Users expect pages that load fast and APIs that answer right away, and that gets harder as the app grows. This is where caching comes in. Redis, an in-memory data structure store, is a good fit for speeding up a Node.js app. Here I cover installing Redis, the basic commands, and using it from Node.js.

What is Redis?

Redis (Remote Dictionary Server) is an open source, in-memory data structure store. It is mostly used as a cache, but it can also act as a database and a message queue. Because it keeps data in RAM, reads and writes are very fast.

Advantages

High performance, because it works in memory.
Several data structures: String, Hash, List, Set and Sorted Set.
Persistence. Even though it is in-memory, it can store data on disk too.
Distributed setups. Redis Cluster gives you high scalability.

Installing Redis locally

Installing Redis on your own machine is simple.

Download and build it:

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

Start the server:

src/redis-server

Start the Redis CLI (command line interface):

src/redis-cli

With that, Redis runs on your machine and the CLI lets you talk to the server.

Installing Redis with Docker

Docker is easier and faster. This command runs Redis in a container:

docker run --name redis -d redis

To stop or start the container again:

docker stop redis
docker start redis

Using Redis from Node.js

To talk to Redis from Node.js we use the redis npm package.

npm install redis

Basic usage:

const redis = require('redis');
const client = redis.createClient();

// When the connection to Redis is established
client.on('connect', function() {
    console.log('Connected to Redis');
});

// When a Redis error occurs
client.on('error', function(err) {
    console.log('Redis error: ' + err);
});

// Write data
client.set('key', 'value', redis.print);

// Read data
client.get('key', function(err, reply) {
    if (err) throw err;
    console.log(reply);
});

Using Redis as a cache

Caching can improve performance a lot for data that is read often and rarely changes. A user profile that you keep fetching from the database is a good example.

Step 1: Fetch the user profile from the database

First, a function that gets the profile from the database:

const getUserProfileFromDB = (userId) => {
    // The database query goes here.
    // We return static data as an example.
    return {
        id: userId,
        name: 'John Doe',
        age: 30
    };
};

Step 2: Add the Redis cache

Now a function that looks in the Redis cache first. If the profile is not there, it reads it from the database and stores it in the cache:

const getUserProfile = (userId, callback) => {
    const cacheKey = `userProfile:${userId}`;

    // Look in Redis first
    client.get(cacheKey, (err, data) => {
        if (err) throw err;

        if (data) {
            // Found in cache
            return callback(JSON.parse(data));
        } else {
            // Not in cache, fetch from the database
            const userProfile = getUserProfileFromDB(userId);
            
            // Save to cache
            client.setex(cacheKey, 3600, JSON.stringify(userProfile));
            
            // Return to the caller
            return callback(userProfile);
        }
    });
};

// Example: fetch a user profile
getUserProfile(1, (profile) => {
    console.log(profile);
});

getUserProfile checks the Redis cache first and returns the cached value if there is one. If not, it reads the profile from the database, saves it to the cache and returns it.

More advanced Redis

Data structures

Redis is not limited to simple key-value pairs. It offers several data structures:

String: simple values.
Hash: field-value pairs.
List: ordered values.
Set: unique values.
Sorted Set: ordered unique values.

You can use these to cache more complex data models.

Pub/Sub

Redis also has a Pub/Sub (publish/subscribe) model. It is common in messaging systems and very useful in microservice architectures.

// Publisher
client.publish('channel', 'message');

// Subscriber
client.subscribe('channel');
client.on('message', (channel, message) => {
    console.log(`Channel: ${channel}, Message: ${message}`);
});

Redis Cluster

For large applications you can build a horizontally scalable setup with Redis Cluster, which spreads the data across several Redis nodes.

Use cases

Session management. Redis is a good place to store and read session data quickly in web apps.

API rate limiting. You can count each user's requests in Redis and block them once they pass a limit.

Real-time data processing. Redis's fast reads and writes help with real-time analytics and live data.

Cache invalidation strategies. These keep stale data in the cache from staying around. They include TTL (Time to Live), manual invalidation and refreshing the cache layer.

TTL (Time to Live)

A TTL makes a cache entry expire automatically after a set time. Old data is cleaned out and fresh data can come in.

// Add a TTL to a cache entry
client.setex('user:1000', 3600, JSON.stringify({ name: 'John Doe', age: 30 }), (err, reply) => {
    if (err) throw err;
    console.log(reply); // OK
});

Manual cache invalidation

With manual invalidation you clear a cached value yourself after a certain operation or event.

const invalidateCache = (cacheKey) => {
    client.del(cacheKey, (err, reply) => {
        if (err) throw err;
        console.log(reply); // 1 (key deleted)
    });
};

// Invalidate the cache after a user update
const updateUserProfile = (userId, newProfile, callback) => {
    // Update the user in the database
    // ...
    
    // Invalidate the cache
    invalidateCache(`userProfile:${userId}`);
    
    callback();
};

Refreshing the cache layer

Here the cache is refreshed automatically, at a fixed interval or after a certain event.

const refreshCache = (cacheKey, dataFetcher, ttl) => {
    dataFetcher((data) => {
        client.setex(cacheKey, ttl, JSON.stringify(data), (err, reply) => {
            if (err) throw err;
            console.log(reply); // OK
        });
    });
};

// Example: refresh the user profile every hour
setInterval(() => {
    refreshCache('userProfile:1000', getUserProfileFromDB.bind(null, 1000), 3600);
}, 3600 * 1000);

Performance tips

A few things to watch when tuning Redis:

Use TTLs. Give cached data a lifetime so old data leaves memory on its own.
Pick the right data structure for what your app needs.
Use Redis Cluster and sharding to spread the load in large applications.

Benchmarking

The Redis Benchmark tool measures how the server performs:

redis-benchmark -q -n 100000

This sends 100,000 requests and prints the results.

Configuration

You can also tune Redis through its configuration. Two important settings:

maxmemory sets the maximum amount of memory Redis can use.

maxmemory 256mb

maxmemory-policy sets what happens when memory is full, for example evicting the least recently used keys (volatile-lru).

maxmemory-policy allkeys-lru