feat: add rename_column support for ALTER TABLE#2391
Draft
william-ch-databricks wants to merge 4 commits intodelta-io:mainfrom
Draft
feat: add rename_column support for ALTER TABLE#2391william-ch-databricks wants to merge 4 commits intodelta-io:mainfrom
william-ch-databricks wants to merge 4 commits intodelta-io:mainfrom
Conversation
This was referenced Apr 14, 2026
a82d990 to
7a91a31
Compare
Collaborator
Author
Range-diff: stack/alter-table-6-drop-column (a82d990 -> 7a91a31)
Reproduce locally: |
73d5ce5 to
628af3c
Compare
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 17, 2026
…2385) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2385/files) to review incremental changes. - [**stack/alter-table-1-refactor-state**](#2385) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2385/files)] - [stack/alter-table-2-supports-data-files](#2386) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2386/files/029d6672081d32caea314dca61f3688e841f3036..50130c30c4378997fbf3d0ef7426ce23992e5c85)] - [stack/alter-table-3-framework-add-column](#2387) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2387/files/50130c30c4378997fbf3d0ef7426ce23992e5c85..eaa5277d025434aa78b79beed1a7cedfe82aa621)] - [stack/alter-table-4-set-nullable](#2388) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2388/files/eaa5277d025434aa78b79beed1a7cedfe82aa621..7506e1273ef349fbf628e4a6db5d0e7c1b4cbd8b)] - [stack/alter-table-5-column-mapping-add](#2389) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2389/files/7506e1273ef349fbf628e4a6db5d0e7c1b4cbd8b..d943ca2945da1ec9c354db57c889d93456381dee)] - [stack/alter-table-6-drop-column](#2390) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2390/files/d943ca2945da1ec9c354db57c889d93456381dee..40e3a1ff9e2915b456d0f0a76a50eb67f27a1ab1)] - [stack/alter-table-7-rename-column](#2391) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2391/files/40e3a1ff9e2915b456d0f0a76a50eb67f27a1ab1..628af3c02767f008a4e3c80a146aec7e5a3ac0b3)] --------- ## What changes are proposed in this pull request? Splits Transaction's snapshot into two concerns: - `read_snapshot_opt: Option<SnapshotRef>` -- the pre-commit table state (None for CREATE TABLE) - `effective_table_config: TableConfiguration` -- the config this commit will produce This separates "what did I read?" (conflict detection, post-commit snapshots) from "what will this commit produce?" (schema, protocol, stats, write context). Write-path call sites read from `effective_table_config`; read-path call sites use `read_snapshot()`. Also adds `should_emit_protocol` / `should_emit_metadata` flags to replace the old `is_create_table()` checks for Protocol/Metadata action emission, and replaces the synthetic pre-commit snapshot in CREATE TABLE with direct `TableConfiguration` construction. This is a pure refactor with no behaviour change. ## How was this change tested? All existing tests pass. Added unit tests for `LogSegment::new_for_version_zero` (valid input, non-zero version rejection, non-commit file rejection).
65ba6c5 to
c09e4fb
Compare
c09e4fb to
4580144
Compare
This was referenced Apr 22, 2026
300d676 to
61fb30d
Compare
ethan-tyler
pushed a commit
to ethan-tyler/delta-kernel-rs
that referenced
this pull request
Apr 23, 2026
…a-io#2386) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2386/files) to review incremental changes. - [stack/alter-table-1-refactor-state](delta-io#2385) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2385/files)] [MERGED] - [**stack/alter-table-2-supports-data-files**](delta-io#2386) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2386/files)] - [stack/alter-table-3-framework-add-column](delta-io#2387) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2387/files/a71c8b8c83d33b1ab882cae984973a76aafc3a31..6dd1b32e8c5163018f2c836f733b58fc4bdc2dc2)] - [stack/alter-table-4-set-nullable](delta-io#2388) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2388/files/6dd1b32e8c5163018f2c836f733b58fc4bdc2dc2..d121f77a73a37f12421c42dc5c4a5626ad2a1e10)] - [stack/alter-table-5-column-mapping-add](delta-io#2389) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2389/files/d121f77a73a37f12421c42dc5c4a5626ad2a1e10..cb5ca4e01024e50ca6b5b0576bbeb9bb6b184d17)] - [stack/alter-table-6-drop-column](delta-io#2390) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2390/files/cb5ca4e01024e50ca6b5b0576bbeb9bb6b184d17..6e91f960109a4d4db662e417780bb4beaac88f5f)] - [stack/alter-table-7-rename-column](delta-io#2391) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2391/files/6e91f960109a4d4db662e417780bb4beaac88f5f..300d676946d785eb195018e2a9ee696c1f6623a6)] - [stack/alter-table-8-operation-string](delta-io#2447) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2447/files/300d676946d785eb195018e2a9ee696c1f6623a6..fee7c029d62cfb55b809ced35c0cb0553ddaf3e7)] - [stack/alter-table-9-is-blind-append](delta-io#2448) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2448/files/fee7c029d62cfb55b809ced35c0cb0553ddaf3e7..3d44b51e0d037c8f33ffbece2d7deecd800d5544)] --------- ## What changes are proposed in this pull request? Introduce a `SupportsDataFiles` marker trait to gate data-file methods at compile time. This prevents future metadata-only transaction types (like `AlterTable`) from accessing methods that add, remove, or configure data files. **Gated behind `SupportsDataFiles` (only `ExistingTable` and `CreateTable`):** | Method | Reason | |--------|--------| | `stats_schema()` | Returns table-specific stats schema for writing files | | `stats_columns()` | Returns column names needing stats collection | | `generate_logical_to_physical()` | Builds partition column transform for file writing | | `get_write_context()` | Returns full write context (target dir, schemas, column mapping) | | `add_files()` | Queues file additions | **Kept on `impl<S> Transaction<S>` (all transaction types):** | Method | Reason | |--------|--------| | `commit()` | Shared commit path; handles empty data gracefully | | `with_data_change()` / `set_data_change()` | ALTER TABLE sets `data_change: false` | | `with_engine_info()` | Applicable to all commits | | `with_commit_info()` | Applicable to all commits | | `with_transaction_id()` | Applicable to all commits | | `with_domain_metadata()` | Schema changes may need domain metadata | | `add_files_schema()` | Read-only accessor; used by `generate_adds()` which handles empty state internally. Kept ungated so it remains available if it becomes dynamic in the future. | | `validate_add_files_stats()` | Internal; handles empty data with early return | | `generate_adds()` | Internal; returns early when no files queued | | `generate_remove_actions()` | Internal; returns early when no files queued | | `generate_dv_update_actions()` | Internal; returns early when no DV updates | | `generate_adds_for_dv_update()` | Internal; only called from `generate_dv_update_actions` | | `build_crc_delta()` | Internal; shared commit path | | `into_committed()` / `into_conflicted()` / `into_retryable()` | Internal; shared commit path | ## How was this change tested? All existing tests pass unchanged. The trait adds compile-time safety with no runtime behavior change. A `compile_fail` doctest will be added in the follow-up PR that introduces `AlterTable` to verify that data file methods are rejected at compile time.
61fb30d to
82bdefd
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2391 +/- ##
==========================================
+ Coverage 88.40% 88.54% +0.13%
==========================================
Files 178 178
Lines 58235 59110 +875
Branches 58235 59110 +875
==========================================
+ Hits 51485 52337 +852
- Misses 4783 4798 +15
- Partials 1967 1975 +8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
3e45b9b to
9d3718e
Compare
0ba1315 to
1f6ffcb
Compare
Implement the SetNullable schema operation which changes a column's nullability from NOT NULL to nullable. If the column is already nullable, this is a no-op. Only the safe direction is allowed (NOT NULL -> nullable) per the Delta protocol.
When column mapping is enabled (mode = name or id), newly added columns via ALTER TABLE ADD COLUMN are now assigned column mapping metadata (unique ID and UUID-based physical name). The maxColumnId table property is updated in the evolved metadata configuration. Reuses the existing assign_field_column_mapping function (now pub(crate)) which handles recursive assignment for nested types.
Implement the DropColumn schema operation which removes a column from the logical schema. Column mapping must be enabled (mode = name or id) because without it, physical Parquet files would still contain the column data. Validates: - Column mapping is enabled - Column exists in the schema - Column is not a partition column - Schema won't be empty after dropping
Adds RenameColumn as a terminal operation in the ALTER TABLE framework: - Renaming type state ensures rename cannot be combined with other operations - Buildable trait + sealed module for shared build() between Modifying and Renaming - Case-insensitive sibling conflict check via sibling_names_at_path - Physical identity (column ID + physical name) preserved via with_name - Runtime defense-in-depth: RenameColumn must be the sole operation Full integration test suite covering add/drop/rename/set_nullable, chaining, sequential alters, checkpoints, time travel, and append-only tables.
1f6ffcb to
4691658
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
What changes are proposed in this pull request?
Adds RenameColumn as a terminal operation in the ALTER TABLE framework:
Full integration test suite covering add/drop/rename/set_nullable, chaining,
sequential alters, checkpoints, time travel, and append-only tables.
How was this change tested?