Skip to content

fix(payload): support sentinel no-op updates in DefaultHoodieRecordPayload#18413

Open
suryaprasanna wants to merge 2 commits intoapache:masterfrom
suryaprasanna:default-hoodie-record-payload
Open

fix(payload): support sentinel no-op updates in DefaultHoodieRecordPayload#18413
suryaprasanna wants to merge 2 commits intoapache:masterfrom
suryaprasanna:default-hoodie-record-payload

Conversation

@suryaprasanna
Copy link
Copy Markdown
Contributor

Describe the issue this Pull Request addresses

This PR updates DefaultHoodieRecordPayload to avoid rewriting records when the incoming payload should not win the merge based on ordering semantics. Instead of returning the current persisted record, it can return a sentinel no-op result so the existing record is preserved without an unnecessary rewrite. The change also makes equal-ordering behavior configurable.

Summary and Changelog

Changes:

  • return SENTINEL from combineAndGetUpdateValue() when the incoming record should not update the persisted record
  • add canProduceSentinel() support in DefaultHoodieRecordPayload
  • add hoodie.payload.update.on.same.ordering.field config with default true
  • add tests covering equal-ordering, older incoming record, newer incoming record, and default behavior

Impact

Changes merge behavior for DefaultHoodieRecordPayload when the incoming record should be ignored. This can reduce unnecessary rewrites and preserve the existing persisted record as-is. Also introduces a new payload config for equal-ordering handling.

Risk Level

medium

This changes merge semantics for the default payload by introducing sentinel-based no-op behavior. Verification was done with targeted unit tests for TestDefaultHoodieRecordPayload, including older, newer, equal-ordering, and default-config cases.

Documentation Update

Config documentation should be updated for hoodie.payload.update.on.same.ordering.field.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@github-actions github-actions bot added the size:M PR with lines of changes in (100, 300] label Mar 29, 2026
@hudi-bot
Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.19%. Comparing base (1eb97b3) to head (a6c721a).

Files with missing lines Patch % Lines
.../hudi/common/model/DefaultHoodieRecordPayload.java 80.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18413      +/-   ##
============================================
- Coverage     68.21%   68.19%   -0.03%     
+ Complexity    27709    27702       -7     
============================================
  Files          2440     2440              
  Lines        134249   134257       +8     
  Branches      16179    16181       +2     
============================================
- Hits          91578    91551      -27     
- Misses        35565    35591      +26     
- Partials       7106     7115       +9     
Flag Coverage Δ
common-and-other-modules 44.32% <80.00%> (-0.01%) ⬇️
hadoop-mr-java-client 45.01% <40.00%> (+0.08%) ⬆️
spark-client-hadoop-common 48.31% <0.00%> (-0.01%) ⬇️
spark-java-tests 48.67% <80.00%> (-0.04%) ⬇️
spark-scala-tests 45.24% <50.00%> (+<0.01%) ⬆️
utilities 38.37% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...g/apache/hudi/common/model/HoodiePayloadProps.java 0.00% <ø> (ø)
.../hudi/common/model/DefaultHoodieRecordPayload.java 78.37% <80.00%> (+1.10%) ⬆️

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

*
* @see DefaultHoodieRecordPayload
*/
public static final String UPDATE_ON_SAME_PAYLOAD_ORDERING_FIELD_PROP_KEY = "hoodie.payload.update.on.same.ordering.field";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The payload class is deprecated, we are not adding new functions to it, should we just add a new payload class impl or just extend the merge mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M PR with lines of changes in (100, 300]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants