Reply to an open minded reader:

The second is even more important, and is about the data model. Redis is not the kind of system where you can insert data and then argue about how to fetch those data in creative ways. Not at all, the whole idea of its data model, and part of the fact that it will be so fast to retrieve your data, is that you need to think in terms of organising your data for fetching. You need to design with the query patterns in mind. In short most of the times your data inside Redis is stored in a way that is natural to query for your use case, that’s why is so fast apart from being in memory, there is no query analysis, optimisation, data reordering. You are just operating on data structures via primitive capabilities offered by those data structures. End of the story.

However, I would add the following perspective on that.

If you do judicious use of your memory, and exploit the fact that Redis sometimes can do a lot with little (see for instance Redis bit operations), and instead of just have a panic attack about your data growing outside the limits of the known universe you try to do your math and consider how much memory commodity servers nowadays have, you’ll discover that there are a tons and one more use cases where it’s ok to have the limit of your computer memory. Often this means tens of millions of active users in one server.

And another point about the data model: remember all those stories about DB denormalisation, and hordes of memcached or Redis farms to cache stuff, and things like that? The reality is that fancy queries are an awesome SQL capability (so incredible that it was hard for all us to escape this warm and comfortable paradigm), but not at scale. So anyway if your data needs to be composed to be served, you are not in good waters.

Source: antirez weblog