Include watched keys and pubsub channel/pattern memory in client memory overhead#3362
Open
enjoy-binbin wants to merge 1 commit intovalkey-io:unstablefrom
Open
Include watched keys and pubsub channel/pattern memory in client memory overhead#3362enjoy-binbin wants to merge 1 commit intovalkey-io:unstablefrom
enjoy-binbin wants to merge 1 commit intovalkey-io:unstablefrom
Conversation
…ry overhead Previously, multiStateMemOverhead() and pubsubMemOverhead() only counted the structure overhead (listNode, watchedKey, hashtable) but not the actual watched_key/channel/pattern robj objects themselves. This led to under-reporting of client memory usage. This was problematic because: 1. If a client watches many keys or watches keys with very large names, this memory was invisible in CLIENT INFO tot-mem, making it hard to identify which client is consuming memory. 2. This memory was incorrectly attributed to used_memory_dataset instead of client memory, making memory analysis misleading. 3. The maxmemory-clients eviction mechanism could not account for this memory, potentially allowing clients to bypass memory limits. The same issues apply to SUBSCRIBE/PSUBSCRIBE/SSUBSCRIBE with large or numerous channel/pattern names. Since each client holds its own robj copy for watched keys and pubsub subscriptions (via incrRefCount on argv), this memory should be attributed to the client for accurate memory accounting. Changes: - Add watched_keys_mem field to multiState struct to track watched key robj memory with O(1) lookup - Add pubsub_object_mem field to ClientPubSubData struct to track channel/pattern robj memory with O(1) lookup The memory counters are updated on add/delete operations, making the overhead calculation O(1) instead of O(N). Signed-off-by: Binbin <[email protected]>
enjoy-binbin
commented
Mar 13, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #3362 +/- ##
============================================
+ Coverage 74.47% 74.53% +0.06%
============================================
Files 130 130
Lines 72719 72728 +9
============================================
+ Hits 54155 54206 +51
+ Misses 18564 18522 -42
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, multiStateMemOverhead() and pubsubMemOverhead() only
counted the structure overhead (listNode, watchedKey, hashtable)
but not the actual watched_key/channel/pattern robj objects
themselves. This led to under-reporting of client memory usage.
This was problematic because:
this memory was invisible in CLIENT INFO tot-mem, making it hard to
identify which client is consuming memory.
of client memory, making memory analysis misleading.
memory, potentially allowing clients to bypass memory limits.
The same issues apply to SUBSCRIBE/PSUBSCRIBE/SSUBSCRIBE with large or
numerous channel/pattern names.
Since each client holds its own robj copy for watched keys and pubsub
subscriptions (via incrRefCount on argv), this memory should be attributed
to the client for accurate memory accounting.
Changes:
robj memory with O(1) lookup
channel/pattern robj memory with O(1) lookup
The memory counters are updated on add/delete operations, making the
overhead calculation O(1) instead of O(N).