Valkey Proxy Proposal #11
Conversation
hwware
force-pushed
the
ValkeyProxy
branch
2 times, most recently
from
67d1a2a to
7b148e5
Compare
| ## Abstract (Required) | ||
|
|
||
| The proposed ValkeyProxy is to provide the features similar to redis-cluster-proxy. | ||
|
|
There was a problem hiding this comment.
The proposed ValkeyProxy is to accept connections from clients and multiplexes them onto connections to the backend server processes. It is horizontally scalable and multi-tenant.
It serves several main purposes beyond simply routing requests to the right processes.
It provides scalability by offloading connection overhead from the backend server processes. It also could hold client connections open even as the server processes are being migrated or restarted. this is very helpful in cloud-native situations, our control over various open source sdks behavior can be extremely limited. For scenarios with lager data volumes, we could provide read-write separation mode through the proxy.
what's more, we could add stateless, secure HTTP connections directly, enhanced connection observability and so on.
ValkeyProxy.md
Outdated
| For some time-consuming or high CPU cost commands, we can make statistics in Proxy side, and for some results, such as min/max/avg/p95/p99 from memtier_benchmark or valkey_benchmark, clients can get timely statistical information. | ||
|
|
||
|
|
||
| 6. Support audit log |
There was a problem hiding this comment.
Why wouldn't we implement audit logs like this into the server though?
There was a problem hiding this comment.
On benefit implemented in proxy side is to retrieve this information from one point, user do not need to fetch the log in different nodes in the cluster mode.
There was a problem hiding this comment.
I'm fairly dubious of proxies in Valkey as a general solution. They do solve problems at scale related to number of connections, but I think many of the problems can also be solved by improving clients.
There are some cool things that could be solved though, like implementing meta-commands (like distributed locks, sorted sets, etc) which is also very interesting. This isn't exactly the same as your proposal, since you seem to be proposing almost API compatibility with the core.
ValkeyProxy.md
Outdated
|
|
||
| ## Design Considerations (Required) | ||
|
|
||
| 1. Support all existing Valkey commands. |
There was a problem hiding this comment.
I would like to bring up that some commands are intrinsically hard to support like lua scripts and pub/sub. Do we need to necessarily implement all commands?
There was a problem hiding this comment.
Even not all, but most is necessary, IMO
|
|
||
| # ValkeyProxy RFC | ||
|
|
||
| ## Abstract (Required) |
There was a problem hiding this comment.
One of the things I'm generally unsure of is whether or not we really a proxy compared to just making clients better. It's sort of been the project of glide to abstract away as much of the differences you have outlined here.
As an aside, we might be able to use the core of glide to implement some of this logic.
ValkeyProxy.md
Outdated
| For some time-consuming or high CPU cost commands, we can make statistics in Proxy side, and for some results, such as min/max/avg/p95/p99 from memtier_benchmark or valkey_benchmark, clients can get timely statistical information. | ||
|
|
||
|
|
||
| 6. Support audit log |
There was a problem hiding this comment.
Why wouldn't we implement audit logs like this into the server though?
ValkeyProxy.md
Outdated
| 8. Provide a connection pool to facilitate client connection. | ||
|
|
||
| Connection pooling can reduce the overhead associated with opening and closing connections and with keeping many | ||
| connections open simultaneously. This overhead not only includes memory needed to handle each new connection, also involves | ||
| CPU overhead to close each connection and open a new one. |
There was a problem hiding this comment.
I think connection pooling is done fairly effectively on the clients today. The only real drawback I've seen is when we have large clusters, so each client has to maintain a very large number of outbound connections
ValkeyProxy.md
Outdated
|
|
||
| For the client's persistent connection, the connection will not actively disconnect, but only returns the error message; If the client is disconnected actively, the connection will be disconnected immediately. | ||
|
|
||
| 9. Support CLIENT LIST etc commands aggregate return: return the client of all proxy nodes. |
There was a problem hiding this comment.
Aggregate commands are cool, we've also heard folks interested in running flushall across the whole cluster. Configuration like config set might also be possible.
ValkeyProxy.md
Outdated
| 7. Provide reading/writing separation features | ||
|
|
||
| In production environments, most scenarios involve more reading requests and less writing requests. By enabling | ||
| read-write separation feature in the Proxy, we can let the primary node focus on processing write operations and let the | ||
| replica handle read operations. It is very suitable for scenarios with relatively large read operations and can reduce the | ||
| pressure on the primary. |
There was a problem hiding this comment.
Ooh, this can go horribly wrong. We talked with a customer that was doing caching and did read-write splits, they were having cache misses on replicas but the writes were going to the primary, but the replica was slow at consuming so it would miss again and trigger another write. I think generally read/write splitting is a big anti-pattern for online transactional workloads like caching or message queues. it makes more sense for analytic workloads, but I don't Valkey really serves much analytics today. Maybe that will change
There was a problem hiding this comment.
The reason we want to implement this feature in ValkeyProxy is:
- This feature (read-write separation) could be enabled or disabled through config parameter. And by integrating with ACL, this feature only applies to some specific users
- For those data with a relatively low update frequency, read misses due to differences in primary and replica data, which in turn lead to multiple writes, are acceptable in some scenarios.
- If data consistency guarantees are a must following the reason 2 case, synchronous between the master and replica can be considered.
Signed-off-by: hwware <[email protected]>
Signed-off-by: hwware <[email protected]>
Signed-off-by: hwware <[email protected]>
Signed-off-by: hwware <[email protected]>
Signed-off-by: hwware <[email protected]>
|
Thank @hwware for putting together this proposal.
My primary concern here is that simulating standalone behavior on a cluster creates a third mode that is neither standalone nor cluster. Losing atomicity is one of the differences, and then there are pub/sub, streams, scan, eval, functions, etc., to consider. I feel we might end up creating a project that is either 10x more complex or has a much narrower scope (no atomicity, no pub/sub, etc.). Networking overhead also plays a huge role in in-memory caching performance. For KV caching workloads, where compatibility is less of a concern, adding a network hop would have a very significant impact. Moreover, the motivations outlined in the RFC don’t necessarily require a proxy to deliver. For example, as @madolson pointed out, Glide already addresses some of these needs with the added potential for end-to-end observability and audit logging. It seems like the real need here might be a better cluster client rather than a proxy layer. A robust client could solve many of these challenges without introducing unnecessary complexity or performance trade-offs. |
No description provided.