Redis as a Database
I’ve been using Redis a lot in the past few years (and even wrote a post about misusing it), mostly as a cache layer between the App and its Database. For some period of time I was actually referring to Redis as a distributed cache solution due to the fact that I was mainly using it for that. And there’s still a misperception of Redis as being “just a cache” among many developers, probably because of its popularity as a cache.
Don’t get me wrong, Redis is a good cache solution, but there is much more we can do with it. Redis is a highly performance scalable distributed data-structure server with advanced NoSQL capabilities, and we are not utilizing those when we’re using it just for caching.
But first thing first, let’s start with a quick recap of Redis:
What is Redis?
Redis is an acronym for “Remote Dictionary Server”: it is an open source in-memory distributed Key-Value store. However, the Value in Redis can hold different kinds of data structures such as strings, hashes, lists, sets, sorted sets (and few others) and not just a s string or a blob like traditional Key-Value stores. Redis does not support any structure query language (SQL), instead each data structure has dedicated set of commands to allow execution of effective atomic operations.
Redis was originally developed by Salvatore Sanfilippo who up until recently was its main contributor, and was designed towards performance from the very early beginning. You can expect sub-millisecond response time with millions of requests per second. Redis also support asynchronous replications and clustering for scaling up.
And yes, Redis core is mostly single threaded when accessing its data (also modern versions do use threads for different things), and requests are executed sequentially.
What is Redis used for?
Redis can be used as a distributed cache, which is probably its most common and popular use case, as a NoSQL Database and as a Message broker (pub\sub mode, similar to Kafka or RabbitMQ).
This post will focus on using Redis as a Database, understand its viable use cases and try to break down the misconception of referring to Redis as just a cache.
Redis Persistence
Redis is an in-memory store, which means all of its data is stored in the server RAM and available only as long as the server is running. Database, on the other end, is usually associated (also it doesn’t have to be) with data durability and persistence storage.
Redis can be configured with different persistence options, but there is a trade-off between performance and persistence level:
- No persistence: Redis can be entirely ephemeral with no persistence at all. Our data is only available while the server is running, and if for some reason the server was terminated or rebooted our data is lost.
- RDB: Redis saves a snapshots of its data at specific time intervals. We can use the snapshot to restore data, but we will lose changes made after the snapshot was created.
- AOF every write operation to Redis is logged, and the log can be used to reconstruct the data. Depending on the how frequently commands are synced to the log, we can expect decrease in server performance.
When talking about Redis persistence, bare in mind that the data is always kept in-memory and requests are executed against the in-memory data regardless of the persistence option we choose (which defines how data is backup to disk, but has nothing to do with accessing data)
Redis as a Database
Cache data is temporary by definition, but we expect data stored in a Database to be long-term persistence. Using an in-memory storage as a Database is not intuitive to many developers (myself included) who are used to think of Databases in terms durability and ACID compliance.
When to use Redis as a Database?
Redis is not a “one size fits all” Database. Not sure that such a Database even exists, but some Databases are more flexible in the variety of use cases they can solve. Consider the following guidelines (with a grain of salt, of course) to evaluate if using Redis as Database fits your needs:
- Performance\speed over durability: depending on the type of data that our App manages and its expected SLO, using high performance Database is sometimes more valuable than using a slower DB that supports long-term persistence.
- Rebuild data quickly: Redis data, or a variation of it, is also stored in other storage(s), and we can rebuild it quickly when we need to. For example, we store raw data in a slower storage while in Redis we store only data aggregations for quick access. We can rebuild data aggregations from the slower storage and save it in Redis whenever we need to.
- Data lost has limited effect: not all data created equal and not all data is missions critical. Depending on our business needs, some of the data might be “expendable” in the sense that even if lost, it will have limited business effect. For example, it is common to store counters in Redis for different uses cases such as limit users access to specific resources (a rate limiter). In the rare condition of losing those counters, users will be able to consume more resources than we initially plan. In most cases, not all of them of course, it has a short term limited business effect.
- Non persistent data: data might be relevant for a limited time duration, usually since it gets updated periodically (not to be confuse with cache data also the idea is the same). Take for example currency conversion rates which are changed daily, IP address geolocation records or a list of potential fraudsters.
- Data size is predictable and limited: Redis data is always kept in memory, even when backup to disk, and RAM is much more expensive than Hard drive (the cost of a 500GB RAM cluster in any Cloud provide environment will convince you of that). We can use in-memory storage Database as long as the amount of data we want to store is limited and we can predict its growth.
When not to use Redis as a Database?
- Critical business data: despite being persistent, critical business data is usually stored in more traditional Databases and not in Redis.
- Complex data queries: data in key-value store can be accessed by its key only. We cannot query records by multiple attributes, not to mentioned more complex queries. Say for example we wanna query our catalog of products for all products that cost less then 50$ and have more then 100 items available for sale — that’s not the type of queries we can execute with Redis.
- Relational data: Redis is a NoSQL database and like other NoSQL Databases its does not build to manage relational data with all kinds of relations between different tables, all kind of constraints and data consistency as Relational Database provides
- Large amount of data: Redis does not fit as a Database if we need to store very large data sets, or expect our data to grow very fast.
Redis Data Structures
In order to actually work with Redis as a Database (or with any other database basically) we need model our data. I’ve mentioned that Redis can hold complex data structures, which provides flexible ways for data modeling.
String
The most basic and intuitive data type, but also very versatile — a String can hold any kind of data like a string, a serialized object, int\float number or even a JPEG image. When using Redis as a cache, cache records are probably stored as Strings.
Lists
Linked-list of strings: pop\push item to the head\tail of a list is very fast O(1), while accessing element at specific index is slow O(N). Redis lists are commonly used for building Queues, but can be useful for storing any series of timeline events like a user recently visit pages or application background jobs.
Sets
Unique collection of strings — adding the same item multiple times to a Set will result in having a single copy of this item in the Set. Sets are useful when we need to store only unique items like site unique visitors, referral pages or exception types from our logs.
Sorted Sets
Hold a Collections of unique strings like Sets, but each element is associated with a score and elements are ordered by their Score. Adding new element is expensive operation O(log(N)) since Sets are kept ordered. Sorted sets are commonly used for building leader boards, but can be used for storing any type of priority data like site most visited page, sold products and so forth.
Hashes
Map of key-value pairs, similar to Java HashMap or C# Dictionary, but the value in Redis hash is also a string and cannot hold (at least not out of the box) nested objects. Hashes are commonly use to represent objects like a User, but we won’t be able to store nested objects like a user Profile.
Wrap it up
When discussing architectural solutions in software, it is important to realize that there are no (almost never) silver bullets, instead there are tradeoffs. The solution that eventually choose depend on how we value those tradeoffs — depending on what we’re building, we might value simplicity over flexibility or performance over data integrity (or the other way around when working on a different project).
The same principles applies to using Redis as a Database. It won’t be the right solution for any project, but it might be exactly the type of Database that you need for a different one.
