Describe the bug
Today, the redis client is built to expect a single endpoint for all operations. This sort of assumes Redis is operating in cluster mode and that the Redis cluster is therefore determining which node the request goes to. However, we will be moving to a primary/replicas configuration to support the use of Celery. At that point, this code is only prepared to work off a single node, the primary.
However, if the API is written such that there are two Redis clients -- one for write operations which points to the primary and one for read operations which points to a replica or a load balancer with replicas behind it -- we can build a more resilient caching system.
Expected behavior
Redis read calls (such as GET) get sent to a Redis client configured to speak with the Redis read-only replicas. Redis write calls (such as SET) get sent to a Redis client configured to speak with the writable Redis primary server.
Actual behavior
All reads and writes currently go to the primary only. If that server becomes overwhelmed, it will trigger a failover. However, the failover will be to a server of similar capacity which will also likely become quickly overwhelmed. Vertical scaling becomes the only option. Whereas, if the replica set is load balanced in the way described above, it is less susceptible to overwhelm and we can scale either/both horizontally and vertically to resolve the issue.
Describe the bug
Today, the redis client is built to expect a single endpoint for all operations. This sort of assumes Redis is operating in cluster mode and that the Redis cluster is therefore determining which node the request goes to. However, we will be moving to a primary/replicas configuration to support the use of Celery. At that point, this code is only prepared to work off a single node, the primary.
However, if the API is written such that there are two Redis clients -- one for write operations which points to the primary and one for read operations which points to a replica or a load balancer with replicas behind it -- we can build a more resilient caching system.
Expected behavior
Redis read calls (such as
GET) get sent to a Redis client configured to speak with the Redis read-only replicas. Redis write calls (such asSET) get sent to a Redis client configured to speak with the writable Redis primary server.Actual behavior
All reads and writes currently go to the primary only. If that server becomes overwhelmed, it will trigger a failover. However, the failover will be to a server of similar capacity which will also likely become quickly overwhelmed. Vertical scaling becomes the only option. Whereas, if the replica set is load balanced in the way described above, it is less susceptible to overwhelm and we can scale either/both horizontally and vertically to resolve the issue.