feat: Block alter table when icebergCompatV3 enabled#2505
feat: Block alter table when icebergCompatV3 enabled#2505dengsh12 wants to merge 6 commits intodelta-io:mainfrom
icebergCompatV3 enabled#2505Conversation
icebergCompatV3 enabled
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2505 +/- ##
=======================================
Coverage 88.50% 88.50%
=======================================
Files 179 179
Lines 59031 59036 +5
Branches 59031 59036 +5
=======================================
+ Hits 52245 52250 +5
Misses 4785 4785
Partials 2001 2001 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| committer: Box<dyn Committer>, | ||
| ) -> DeltaResult<AlterTableTransaction> { | ||
| let table_config = self.snapshot.table_configuration(); | ||
| // We don't support ALTER TABLE on tables with icebergCompatV3 enabled yet. See |
There was a problem hiding this comment.
Note: We could put the validation on constructor of the AlterTableTransactionBuilder(i.e. add a try_new) and fail early. But that breaks all callers by a DeltaResult(currently constructor of AlterTableTransactionBuilder never fails). As this is a temporal limitation, feel like not need to break callers here
70b7fb4 to
ffd8ecb
Compare
ffd8ecb to
cae1ea9
Compare
icebergCompatV3 enabledicebergCompatV3 enabled
icebergCompatV3 enabledicebergCompatV3 enabled
d85955c to
3f1154c
Compare
…#2475) ## 🥞 Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2475/files) to review incremental changes. - [**stack/add-v3-feature**](delta-io#2475) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2475/files)] - stack/write-nested-id - [stack/write-part-when-ice3](delta-io#2504) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2504/files/e1a4b559271b72323a9b560aa456db49276b63e1..9f777f08559a7f0b7e014756182fbcb1ae788996)] - [stack/block-alter-table](delta-io#2505) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2505/files/9f777f08559a7f0b7e014756182fbcb1ae788996..ffd8ecb8ce71681530b92f3683977bd58b9d57b0)] --------- ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2475/files) to review incremental changes. - [**stack/add-v3-feature**](delta-io#2475) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2475/files)] - stack/write-nested-id - [stack/write-part-when-ice3](delta-io#2504) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2504/files/e1a4b559271b72323a9b560aa456db49276b63e1..5442d0979cbe36fc2eb1e85bf0a79bc01c30e121)] --------- ## What changes are proposed in this pull request? Add `icebergCompatV3` and it's `FeatureInfo` ## How was this change tested? Tested happy/unhappy paths for `icebergCompatV3`: 1. Error when row tracking not enabled 2. Error when column mapping not enabled 3. Error when `icebergCompatV1/V2` enabled 4. `icebergCompatV3` enablement follows the corresponding table property 5. Write operation is blocked 6. Success when all requirements are met
3f1154c to
ff634da
Compare
| // [`crate::table_features::ICEBERG_COMPAT_V3_INFO`] for the tracking issue. | ||
| if table_config.is_feature_enabled(&TableFeature::IcebergCompatV3) { | ||
| return Err(Error::unsupported( | ||
| "ALTER TABLE is not supported on tables with icebergCompatV3 enabled", |
There was a problem hiding this comment.
This error makes it seem like ALTER TABLE is fundamentally not supported on IcebergCompatV3 tables.
The reality is: ALTER TABLE is not yet supported on IcebergCompatV3 tables.
Please add yet there.
There was a problem hiding this comment.
Good point! Clarified: added yet
scottsand-db
left a comment
There was a problem hiding this comment.
LGTM but please implement my feedback to clarify and improve the error message
ff634da to
bfb086b
Compare
562605f to
cd4bf3f
Compare
…lta-io#2504) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2504/files) to review incremental changes. - [stack/add-v3-feature](delta-io#2475) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2475/files)] [MERGED] - [**stack/write-part-when-ice3**](delta-io#2504) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2504/files)] - [stack/block-alter-table](delta-io#2505) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2505/files/15d151f8b271a72cb421ccfa81cda7d5887d1b37..562605f3911acd99854efae7940deb9344e7ee55)] - [stack/write-nested-id](delta-io#2508) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2508/files/562605f3911acd99854efae7940deb9344e7ee55..766ace41c2176c546f533be4ff0d7a9aa24ae001)] --------- ## What changes are proposed in this pull request? Materialize partition values when icebergCompatV3 enabled. ## How was this change tested? 1. When `icebergCompatV3` enabled, partition columns are contained into `physical_write_schema` 2. When `icebergCompatV3` supported but not enabled, partition columns are not contained into `physical_write_schema` Note: other related cases(e.g. `MaterializePartitionColumns`) are already covered by existing tests so we are not adding tests here
🥞 Stacked PR
Use this link to review incremental changes.
What changes are proposed in this pull request?
Block alter table when icebergCompatV3 enabled. Alter table transactions are expected to build from
AlterTableTransactionBuilder::build, thus we block that function on icebergCompatV3 tables.How was this change tested?
Integration test: create an
icebergCompatV3table and validate the error.Note: Successful paths have been covered by existing tests so we are not adding test for successful path here