Skip to content

Optimized PartitionedOutput staging hub#1799

Open
yingsu00 wants to merge 15 commits intooss-mainfrom
optimized_partitionedoutput
Open

Optimized PartitionedOutput staging hub#1799
yingsu00 wants to merge 15 commits intooss-mainfrom
optimized_partitionedoutput

Conversation

@yingsu00
Copy link
Copy Markdown
Collaborator

@yingsu00 yingsu00 commented Mar 11, 2026

This PR is to track the commits for the optimized PartitionedOutput operator. Related new PRs shall be submitted to optimized_partitionedoutput branch, and then alchemy merge to oss-main.

Review history of the first two commits can be found in
feat: Introducing PartitionedVector #1596
feat: Add PartitionedRowVector implementation #1770

@yingsu00 yingsu00 requested a review from majetideepak as a code owner March 11, 2026 14:56
@yingsu00 yingsu00 removed the request for review from majetideepak March 11, 2026 14:56
@yingsu00
Copy link
Copy Markdown
Collaborator Author

alchemy link 86db93b

@prestodb-ci
Copy link
Copy Markdown
Collaborator

Added new rebase item:

@yingsu00
Copy link
Copy Markdown
Collaborator Author

alchemy link 6dd3661

@prestodb-ci
Copy link
Copy Markdown
Collaborator

The following unexpired item was removed at 2026-03-10T12:01:55Z by @yingsu00 via #1799 (comment):

Added new rebase item:

@yingsu00
Copy link
Copy Markdown
Collaborator Author

alchemy link 2706c1e

@prestodb-ci
Copy link
Copy Markdown
Collaborator

The following unexpired item was removed at 2026-03-12T13:19:10Z by @yingsu00 via #1799 (comment):

Added new rebase item:

@unidevel
Copy link
Copy Markdown
Collaborator

alchemy link 86db93b,6dd3661c7afef52c42ed1c5ca83c9e57e21ec2b3,2706c1e80f9463bd4fdd805e296839f964437ca3

@prestodb-ci
Copy link
Copy Markdown
Collaborator

Failed to add new rebase item:

The new rebase item overlaps with the following existing item:

Please double check your input and retry.

@unidevel
Copy link
Copy Markdown
Collaborator

alchemy link 86db93b,6dd3661c7afef52c42ed1c5ca83c9e57e21ec2b3,2706c1e80f9463bd4fdd805e296839f964437ca3 @2026-03-10T12:01:55Z

@prestodb-ci
Copy link
Copy Markdown
Collaborator

Failed to add new rebase item:

The new rebase item overlaps with the following existing item:

Please double check your input and retry.

@unidevel
Copy link
Copy Markdown
Collaborator

alchemy link 86db93b,6dd3661c7afef52c42ed1c5ca83c9e57e21ec2b3,2706c1e80f9463bd4fdd805e296839f964437ca3 @2026-03-12T13:19:10Z

@prestodb-ci
Copy link
Copy Markdown
Collaborator

The following unexpired item was removed at 2026-03-12T13:19:10Z by @unidevel via #1799 (comment):

Added new rebase item:

…fferManager listeners

Pass an OutputBufferManager-backed listener factory into
PrestoIterativePartitioningSerializer so the optimized path uses the same
listener source as normal PartitionedOutput. Create per-partition listeners
during flush, set the checksum bit only when a listener is present, and
compute the page checksum only for PrestoOutputStreamListener instances.
Also add tests that verify checksum headers are written and that the
serialized pages round-trip through the standard deserializer.
…rting

- add explicit simple-schema benchmark cases by type and column count
- register normal and optimized runs as separate named benchmark cases
- make `dictPct` apply per generated vector and recurse into nested types
- generate benchmark input vectors directly with optional nulls
- replace ad hoc flat input generation with explicit input specs
- return `ExchangeRunStats` from benchmark runs and centralize query config
- group printed results by dataset with normal vs. optimized stats
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants