Developers increasingly need a variety of datastores for their projects -- no one database can serve all the needs of a modern, scalable application. For example, an e-commerce app might store its valuable transaction data in a relational database while user session information is stored in a key-value store because it changes often and needs to be accessed quickly. This is a common pattern across many app types, and the need for a key-value store is especially acute. Today, we are pleased to announce the beta of Heroku Key-Value Store, joining Heroku Postgres as our second data service.
We have deep experience with Redis; internally at Heroku, we use Redis extensively as a queue, as a cache, and in a variety of other roles to complement Postgres and to build Heroku. Redis’ high throughput, in-memory architecture and simplicity of interface via key-value semantics makes it ideal for building data-driven applications. The beta of our Heroku Key-Value Store service adds to those native strengths a number of developer experience features that make it easy to utilize Redis at any scale, including performance analytics and metrics logs. With these added usability features and the reliability developers have come to expect from Heroku, we believe you can build more powerful data-driven apps than ever before.
Why Redis
Redis is an in-memory, key-value datastore that has excellent support for all of the official languages on Heroku. It provides a few powerful data types and has a focus on maximizing throughput. By staying simple, and cutting away all of the complexity of a relational database system, Redis has been able to deliver incredible performance results. The entire dataset lives in RAM, and because there is no support for expensive table joins or other aggregate computations, response time is always fast. This simplicity comes at a cost. Redis lacks the durability guarantees of a more fully-featured database. We recommend you always store your valuable data in another system, and use Redis in a supporting role.
Use Cases
We use Redis extensively at Heroku, from monitoring our Heroku Postgres product and router logging to queueing up the build process for customers. The flexibility that Redis provides has allowed us to build a distributed platform at scale. This flexibility can be attributed to a great set of primitives that allows the application developer to build against unique use cases very quickly including:
- Job Queues: Queues are used extensively in web development to separate long-running tasks from the normal request-response cycle of the web server. Redis has primitives like BLPOP that allow workers to wait for jobs to get pushed onto the queue. This behavior has been popularized in libraries like Sidekiq.
- API Rate Limiting: If you have an API that provides data to developers, you might consider limiting the number of API calls that can be made at any one time so that your backend systems aren’t overrun. Projects like redis-throttle allow Ruby developers to drop in rate limiting with little fuss.
- Session Storage: Every web app that wants to track users needs to store session information because HTTP is a stateless protocol. Wherever the session is being saved, it needs to be looked up on every HTTP request. Redis makes a great data store for session data because of its performance characteristics. Projects like connect-redis for Node.js adds this ability via middleware.
- Share Resources Between Processes: Let’s say you’ve got multiple dynos trying to get exclusive access to the same resource. You can use a distributed lock in Redis so that only one process will ever have access at any one time. Redis has documentation and a recommended algorithm called Redlock with implementations in Ruby, Python, PHP, and Java.
- Caching: Some data needs to be accessed quickly and very often. This is one of the sweet spots for Redis. Many web frameworks and runtimes, like Django, Rails, and Node.js, have drop-in extensions to use Redis as a cache.
These are but a small sampling of the use cases that Redis can help you solve in your application. With some quick searching you can find many more situations where Redis creates a lot of value including from the author of Redis himself, Salvatore 'antirez' Sanfilippo.
Making Redis Even Better
We wanted to take Heroku Key-Value Store beyond just providing it as a service. We wanted to bring the same level of experience that developers expect from Heroku products and build that into something worthy of being called Heroku Key-Value Store. We’ve created a number of features that make it easy to operate and maintain Redis at any scale. The highlight of these features that we’re releasing at the start of the beta period includes our performance analytics and metrics logs.
Our performance analytics comes as part of the web interface for Heroku Key-Value Store. Developers will get insight into how their Redis is performing against its limits, like the amount of memory being used and the number of connections over time. By having the historical information of how your Redis instance is performing, you’ll gain the ability to diagnose application issues or scale your Redis instance further.
The other feature we’re launching during this beta is our metrics logs. As applications get sufficiently large, developers tend to have their own monitoring frameworks like Librato to understand what’s happening across all aspects of the architecture. The metrics logs provide the same information that you’d see in performance analytics but with much more granularity. To see these events, you can find them in your application’s log stream:
heroku logs -p heroku-redis -t
Creating a Heroku Key-Value Store Instance
Starting today, you can add the beta version of Heroku Key-Value Store to your application architecture:
heroku addons:create heroku-redis:test
What you’ll get is a Redis instance with 20MB of RAM and 20 concurrent client connections. On top of that, you’ll get a REDIS_URL config var for your application to consume. If you are already using REDIS_URL in your application, the command will add a color config var like HEROKU_REDIS_YELLOW_URL. Heroku Key-Value Store is free for the duration of the public beta and includes access to the Performance Metrics page in the Heroku Key-Value Store dashboard. We recommend that developers wait for general availability before deploying Heroku Key-Value Store in a production environment.
Building A Great Redis Experience
During the public beta, we’re very interested in collecting feedback on Heroku Key-Value Store. We are keenly focused on bringing an integrated Heroku developer experience across all of our data products before we reach general availability, so please try the beta and share your thoughts with us. Once we get to GA, we’ll provide more details on the plan and pricing options. In the meantime, if you have any ideas, comments, or support questions, the Redis team would love to hear from you.