Skip to content

Commit f90025c

Browse files
craig[bot]techcodie
andcommitted
Merge #158602
158602: changefeedccl: improve changefeed.default_range_distribution_strategy… r=andyyang890 a=techcodie … description Update the cluster setting description to be clearer and more informative: - Explain what each option (default vs balanced_simple) actually does - Clarify the relationship with locality restrictions - Mention the per-changefeed override option - Note that this setting replaces the deprecated changefeed.balance_range_distribution.enable This improves the auto-generated documentation on the cluster settings page and helps users understand when and how to use this setting. Fixes #101219 Epic: None Co-authored-by: Ansh <[email protected]>
2 parents 432dd73 + 03dac65 commit f90025c

File tree

3 files changed

+8
-6
lines changed

3 files changed

+8
-6
lines changed

docs/generated/settings/settings-for-tenants.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ changefeed.aggregator.flush_jitter float 0.1 jitter aggregator flushes as a frac
1212
changefeed.backfill.concurrent_scan_requests integer 0 number of concurrent scan requests per node issued during a backfill application
1313
changefeed.backfill.scan_request_size integer 524288 the maximum number of bytes returned by each scan request application
1414
changefeed.batch_reduction_retry.enabled (alias: changefeed.batch_reduction_retry_enabled) boolean false if true, kafka changefeeds upon erroring on an oversized batch will attempt to resend the messages with progressively lower batch sizes application
15-
changefeed.default_range_distribution_strategy enumeration default configures how work is distributed among nodes for a given changefeed. for the most balanced distribution, use `balanced_simple`. changing this setting will not override locality restrictions. this setting can be overridden by the changefeed option `range_distribution_strategy` [default = 0, balanced_simple = 1] application
15+
changefeed.default_range_distribution_strategy enumeration default controls how changefeed work is distributed across nodes. 'default' defers to DistSQL for node selection and work distribution. 'balanced_simple' uses DistSQL for node selection but then attempts to evenly distribute ranges across those selected nodes for better load balancing. this setting does not override locality restrictions and can be overridden per-changefeed using the 'range_distribution_strategy' option. [default = 0, balanced_simple = 1] application
1616
changefeed.event_consumer_worker_queue_size integer 16 if changefeed.event_consumer_workers is enabled, this setting sets the maxmimum number of events which a worker can buffer application
1717
changefeed.event_consumer_workers integer 0 the number of workers to use when processing events: <0 disables, 0 assigns a reasonable default, >0 assigns the setting value. for experimental/core changefeeds and changefeeds using parquet format, this is disabled application
1818
changefeed.fast_gzip.enabled boolean true use fast gzip implementation application

docs/generated/settings/settings.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
<tr><td><div id="setting-changefeed-backfill-concurrent-scan-requests" class="anchored"><code>changefeed.backfill.concurrent_scan_requests</code></div></td><td>integer</td><td><code>0</code></td><td>number of concurrent scan requests per node issued during a backfill</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
1818
<tr><td><div id="setting-changefeed-backfill-scan-request-size" class="anchored"><code>changefeed.backfill.scan_request_size</code></div></td><td>integer</td><td><code>524288</code></td><td>the maximum number of bytes returned by each scan request</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
1919
<tr><td><div id="setting-changefeed-batch-reduction-retry-enabled" class="anchored"><code>changefeed.batch_reduction_retry.enabled<br />(alias: changefeed.batch_reduction_retry_enabled)</code></div></td><td>boolean</td><td><code>false</code></td><td>if true, kafka changefeeds upon erroring on an oversized batch will attempt to resend the messages with progressively lower batch sizes</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
20-
<tr><td><div id="setting-changefeed-default-range-distribution-strategy" class="anchored"><code>changefeed.default_range_distribution_strategy</code></div></td><td>enumeration</td><td><code>default</code></td><td>configures how work is distributed among nodes for a given changefeed. for the most balanced distribution, use `balanced_simple`. changing this setting will not override locality restrictions. this setting can be overridden by the changefeed option `range_distribution_strategy` [default = 0, balanced_simple = 1]</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
20+
<tr><td><div id="setting-changefeed-default-range-distribution-strategy" class="anchored"><code>changefeed.default_range_distribution_strategy</code></div></td><td>enumeration</td><td><code>default</code></td><td>controls how changefeed work is distributed across nodes. &#39;default&#39; defers to DistSQL for node selection and work distribution. &#39;balanced_simple&#39; uses DistSQL for node selection but then attempts to evenly distribute ranges across those selected nodes for better load balancing. this setting does not override locality restrictions and can be overridden per-changefeed using the &#39;range_distribution_strategy&#39; option. [default = 0, balanced_simple = 1]</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
2121
<tr><td><div id="setting-changefeed-event-consumer-worker-queue-size" class="anchored"><code>changefeed.event_consumer_worker_queue_size</code></div></td><td>integer</td><td><code>16</code></td><td>if changefeed.event_consumer_workers is enabled, this setting sets the maxmimum number of events which a worker can buffer</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
2222
<tr><td><div id="setting-changefeed-event-consumer-workers" class="anchored"><code>changefeed.event_consumer_workers</code></div></td><td>integer</td><td><code>0</code></td><td>the number of workers to use when processing events: &lt;0 disables, 0 assigns a reasonable default, &gt;0 assigns the setting value. for experimental/core changefeeds and changefeeds using parquet format, this is disabled</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>
2323
<tr><td><div id="setting-changefeed-fast-gzip-enabled" class="anchored"><code>changefeed.fast_gzip.enabled</code></div></td><td>boolean</td><td><code>true</code></td><td>use fast gzip implementation</td><td>Basic/Standard/Advanced/Self-Hosted</td></tr>

pkg/ccl/changefeedccl/changefeed_dist.go

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -362,10 +362,12 @@ var rangeDistributionStrategyStrings = map[clusterSettingRangeDistributionType]s
362362
var RangeDistributionStrategy = settings.RegisterEnumSetting(
363363
settings.ApplicationLevel,
364364
"changefeed.default_range_distribution_strategy",
365-
"configures how work is distributed among nodes for a given changefeed. "+
366-
"for the most balanced distribution, use `balanced_simple`. changing this setting "+
367-
"will not override locality restrictions. this setting can be overridden by the "+
368-
"changefeed option `range_distribution_strategy`",
365+
"controls how changefeed work is distributed across nodes. "+
366+
"'default' defers to DistSQL for node selection and work distribution. "+
367+
"'balanced_simple' uses DistSQL for node selection but then attempts to evenly "+
368+
"distribute ranges across those selected nodes for better load balancing. "+
369+
"this setting does not override locality restrictions and can be overridden "+
370+
"per-changefeed using the 'range_distribution_strategy' option.",
369371
metamorphic.ConstantWithTestChoice("default_range_distribution_strategy",
370372
string(changefeedbase.ChangefeedRangeDistributionStrategyDefault),
371373
string(changefeedbase.ChangefeedRangeDistributionStrategyBalancedSimple)),

0 commit comments

Comments
 (0)