feat: derive ALTER TABLE commit operation string from schema ops#2447
Draft
william-ch-databricks wants to merge 5 commits intodelta-io:mainfrom
Draft
feat: derive ALTER TABLE commit operation string from schema ops#2447william-ch-databricks wants to merge 5 commits intodelta-io:mainfrom
william-ch-databricks wants to merge 5 commits intodelta-io:mainfrom
Conversation
This was referenced Apr 22, 2026
This was referenced Apr 22, 2026
william-ch-databricks
force-pushed
the
stack/alter-table-8-operation-string
branch
6 times, most recently
from
fee7c02 to
8f86aa3
Compare
ethan-tyler
pushed a commit
to ethan-tyler/delta-kernel-rs
that referenced
this pull request
Apr 23, 2026
…a-io#2386) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2386/files) to review incremental changes. - [stack/alter-table-1-refactor-state](delta-io#2385) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2385/files)] [MERGED] - [**stack/alter-table-2-supports-data-files**](delta-io#2386) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2386/files)] - [stack/alter-table-3-framework-add-column](delta-io#2387) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2387/files/a71c8b8c83d33b1ab882cae984973a76aafc3a31..6dd1b32e8c5163018f2c836f733b58fc4bdc2dc2)] - [stack/alter-table-4-set-nullable](delta-io#2388) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2388/files/6dd1b32e8c5163018f2c836f733b58fc4bdc2dc2..d121f77a73a37f12421c42dc5c4a5626ad2a1e10)] - [stack/alter-table-5-column-mapping-add](delta-io#2389) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2389/files/d121f77a73a37f12421c42dc5c4a5626ad2a1e10..cb5ca4e01024e50ca6b5b0576bbeb9bb6b184d17)] - [stack/alter-table-6-drop-column](delta-io#2390) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2390/files/cb5ca4e01024e50ca6b5b0576bbeb9bb6b184d17..6e91f960109a4d4db662e417780bb4beaac88f5f)] - [stack/alter-table-7-rename-column](delta-io#2391) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2391/files/6e91f960109a4d4db662e417780bb4beaac88f5f..300d676946d785eb195018e2a9ee696c1f6623a6)] - [stack/alter-table-8-operation-string](delta-io#2447) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2447/files/300d676946d785eb195018e2a9ee696c1f6623a6..fee7c029d62cfb55b809ced35c0cb0553ddaf3e7)] - [stack/alter-table-9-is-blind-append](delta-io#2448) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2448/files/fee7c029d62cfb55b809ced35c0cb0553ddaf3e7..3d44b51e0d037c8f33ffbece2d7deecd800d5544)] --------- ## What changes are proposed in this pull request? Introduce a `SupportsDataFiles` marker trait to gate data-file methods at compile time. This prevents future metadata-only transaction types (like `AlterTable`) from accessing methods that add, remove, or configure data files. **Gated behind `SupportsDataFiles` (only `ExistingTable` and `CreateTable`):** | Method | Reason | |--------|--------| | `stats_schema()` | Returns table-specific stats schema for writing files | | `stats_columns()` | Returns column names needing stats collection | | `generate_logical_to_physical()` | Builds partition column transform for file writing | | `get_write_context()` | Returns full write context (target dir, schemas, column mapping) | | `add_files()` | Queues file additions | **Kept on `impl<S> Transaction<S>` (all transaction types):** | Method | Reason | |--------|--------| | `commit()` | Shared commit path; handles empty data gracefully | | `with_data_change()` / `set_data_change()` | ALTER TABLE sets `data_change: false` | | `with_engine_info()` | Applicable to all commits | | `with_commit_info()` | Applicable to all commits | | `with_transaction_id()` | Applicable to all commits | | `with_domain_metadata()` | Schema changes may need domain metadata | | `add_files_schema()` | Read-only accessor; used by `generate_adds()` which handles empty state internally. Kept ungated so it remains available if it becomes dynamic in the future. | | `validate_add_files_stats()` | Internal; handles empty data with early return | | `generate_adds()` | Internal; returns early when no files queued | | `generate_remove_actions()` | Internal; returns early when no files queued | | `generate_dv_update_actions()` | Internal; returns early when no DV updates | | `generate_adds_for_dv_update()` | Internal; only called from `generate_dv_update_actions` | | `build_crc_delta()` | Internal; shared commit path | | `into_committed()` / `into_conflicted()` / `into_retryable()` | Internal; shared commit path | ## How was this change tested? All existing tests pass unchanged. The trait adds compile-time safety with no runtime behavior change. A `compile_fail` doctest will be added in the follow-up PR that introduces `AlterTable` to verify that data file methods are rejected at compile time.
william-ch-databricks
force-pushed
the
stack/alter-table-8-operation-string
branch
from
8f86aa3 to
12d3aca
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2447 +/- ##
==========================================
+ Coverage 88.40% 88.54% +0.14%
==========================================
Files 178 178
Lines 58235 59152 +917
Branches 58235 59152 +917
==========================================
+ Hits 51485 52379 +894
- Misses 4783 4798 +15
- Partials 1967 1975 +8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
william-ch-databricks
force-pushed
the
stack/alter-table-8-operation-string
branch
3 times, most recently
from
0f74db3 to
540a036
Compare
william-ch-databricks
force-pushed
the
stack/alter-table-8-operation-string
branch
4 times, most recently
from
422b399 to
2ef3fb3
Compare
Implement the SetNullable schema operation which changes a column's nullability from NOT NULL to nullable. If the column is already nullable, this is a no-op. Only the safe direction is allowed (NOT NULL -> nullable) per the Delta protocol.
When column mapping is enabled (mode = name or id), newly added columns via ALTER TABLE ADD COLUMN are now assigned column mapping metadata (unique ID and UUID-based physical name). The maxColumnId table property is updated in the evolved metadata configuration. Reuses the existing assign_field_column_mapping function (now pub(crate)) which handles recursive assignment for nested types.
Implement the DropColumn schema operation which removes a column from the logical schema. Column mapping must be enabled (mode = name or id) because without it, physical Parquet files would still contain the column data. Validates: - Column mapping is enabled - Column exists in the schema - Column is not a partition column - Schema won't be empty after dropping
Adds RenameColumn as a terminal operation in the ALTER TABLE framework: - Renaming type state ensures rename cannot be combined with other operations - Buildable trait + sealed module for shared build() between Modifying and Renaming - Case-insensitive sibling conflict check via sibling_names_at_path - Physical identity (column ID + physical name) preserved via with_name - Runtime defense-in-depth: RenameColumn must be the sole operation Full integration test suite covering add/drop/rename/set_nullable, chaining, sequential alters, checkpoints, time travel, and append-only tables.
Replaces the hardcoded "ALTER TABLE" operation string with a per-op derived string that matches delta-spark (e.g. "ADD COLUMNS", "ADD COLUMNS + CHANGE COLUMNS"). The operation name is emitted by `apply_schema_operations` via `SchemaEvolutionResult.operation`; distinct names are joined with " + " in first-occurrence order.
william-ch-databricks
force-pushed
the
stack/alter-table-8-operation-string
branch
from
2ef3fb3 to
f66290e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
What changes are proposed in this pull request?
Replaces the hardcoded `"ALTER TABLE"` commit operation string with a per-op derived string matching delta-spark:
The string is produced by `apply_schema_operations` via `SchemaEvolutionResult.operation` and propagated through the ALTER transaction.
How was this change tested?
Integration test `alter_commit_operation` (rstest, 5 cases) commits each single-op variant + a chained variant and asserts the recorded `commitInfo.operation` via `read_commit_operation`. Unit tests in `schema_evolution.rs` cover dedup of repeated ops and the empty-ops edge case.