feat!: Add feature `icebergCompatV3` and it's `FeatureInfo` by dengsh12 · Pull Request #2475 · delta-io/delta-kernel-rs

dengsh12 · 2026-04-27T23:27:24Z

🥞 Stacked PR

Use this link to review incremental changes.

stack/add-v3-feature [Files changed]
- stack/write-nested-id
- stack/write-part-when-ice3 [Files changed]
  - stack/block-alter-table [Files changed]

Stacked PR

Use this link to review incremental changes.

stack/add-v3-feature [Files changed]
- stack/write-nested-id
- stack/write-part-when-ice3 [Files changed]

What changes are proposed in this pull request?

Add icebergCompatV3 and it's FeatureInfo

How was this change tested?

Tested happy/unhappy paths for icebergCompatV3:

Error when row tracking not enabled
Error when column mapping not enabled
Error when icebergCompatV1/V2 enabled
icebergCompatV3 enablement follows the corresponding table property
Write operation is blocked
Success when all requirements are met

codecov · 2026-04-27T23:31:59Z

Codecov Report

❌ Patch coverage is 98.64865% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.54%. Comparing base (b61bf96) to head (e1a4b55).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
kernel/src/table_properties/deserialize.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2475      +/-   ##
==========================================
+ Coverage   88.47%   88.54%   +0.07%     
==========================================
  Files         178      178              
  Lines       58807    58861      +54     
  Branches    58807    58861      +54     
==========================================
+ Hits        52028    52120      +92     
+ Misses       4793     4754      -39     
- Partials     1986     1987       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

scottsand-db

Looks great! minor comments

scottsand-db · 2026-04-28T16:15:29Z

Do we need this line here btw? Can we clean this up? Since Enabled already covers this?

Good catch! Cleared existing repeated validations

scottsand-db · 2026-04-28T16:15:41Z

scottsand-db · 2026-04-28T16:15:55Z

            }
            Ok(())
        }),
        FeatureRequirement::NotSupported(TableFeature::DeletionVectors),


Requires V2 and V3 to be NOT supported

Added(validate V2V3not enabled)

scottsand-db · 2026-04-28T16:16:03Z

            }
            Ok(())
        }),
        FeatureRequirement::NotEnabled(TableFeature::IcebergCompatV1),


Also requires V3 to be not supported

Added(validate V3not enabled)

scottsand-db · 2026-04-28T16:17:18Z

+        FeatureRequirement::Enabled(TableFeature::RowTracking),
+        // V1/V2 may remain in `writerFeatures` (supported) as long as they are not active --
+        // hence `NotEnabled` rather than `NotSupported`.
+        FeatureRequirement::NotEnabled(TableFeature::IcebergCompatV1),


They can be supported? just not enabled? -- The protocol says Require that IcebergCompatV1 and IcebergCompatV2 are not active on the table --- so, unfortunately, we have 3 adjectives here -- supported, enabled, active.

Can you please work with Hao to clear this up?

They can be supported? just not enabled?

According to other impls yes. And yeah the word active confused me a little...

Can you please work with Hao to clear this up?

Sure! Let me directly open a PR to change the word active to enabled

Update from Hao: https://github.com/delta-io/delta/blob/master/PROTOCOL.md#active-features seems active is also a formal delta word

I remember now: active == enabled. // sigh, wish the entire protocol was aligned on this

scottsand-db · 2026-04-28T16:20:47Z

+    /// explicitly. `name`/`id` modes need a column-mapping-annotated schema (otherwise
+    /// `TableConfiguration::try_new` rejects the metadata for missing per-field annotations);
+    /// this helper swaps the schema accordingly.
+    fn create_mock_table_config_with_cm(


Do we have existing create_mock_table functions? I see

create_mock_table_config

create_mock_table_config_with_version

create_mock_table_config_with_cm

Can you please create an issue and add a TODO comment here? // TODO(#12345): Consolidate ...

Also -- can you please create an IcebergCompatV3 tracking issue? See #2393 as an example (you can ignore the migration guide)

Added #2492 for tracking, #2491 for consolidating the mock TableConfiguration

dengsh12 · 2026-04-28T22:12:45Z

+    #[case::v3_rejects_v1(
+        TableFeature::IcebergCompatV3,
+        TableFeature::IcebergCompatV1,
+        "requires 'icebergCompatV1' to not be enabled"
+    )]
+    #[case::v3_rejects_v2(
+        TableFeature::IcebergCompatV3,
+        TableFeature::IcebergCompatV2,
+        "requires 'icebergCompatV2' to not be enabled"
+    )]


v3 rejects v1, v2 is also covered by another test: test_iceberg_compat_v3_feature_requirements, add the tests again here just to make test_iceberg_compat_mutual_exclusion looks more complete

scottsand-db

LGTM! TY

scottsand-db · 2026-04-29T21:43:46Z

+// Spec: <https://github.com/delta-io/delta/blob/master/protocol_rfcs/iceberg-compat-v3.md>
+//
+// TODO: Implement the write-side requirements for IcebergCompatV3.
+// Note: unlike V1/V2, V3 intentionally permits DeletionVectors per the RFC -- no


this can be moved into the feature_requirements below IMO. also try and avoid -- usually looks like AI

Moved and removed --

scottsand-db · 2026-04-29T21:44:21Z

+        FeatureRequirement::NotEnabled(TableFeature::IcebergCompatV2),
+    ],
+    kernel_support: KernelSupport::NotSupported,
+    enablement_check: EnablementCheck::EnabledIf(|props| {


Should these enablement checks also be double checking that the feature is supported? or is that implied?

Enabled includes supported

/// Generic method to check if a feature is enabled. /// /// A feature is enabled if: /// 1. It is supported in the protocol /// 2. The enablement check passes #[internal_api] pub(crate) fn is_feature_enabled(&self, feature: &TableFeature) -> bool { if !self.is_feature_supported(feature) { return false; } match feature.info().enablement_check { EnablementCheck::AlwaysIfSupported => true, EnablementCheck::EnabledIf(check_fn) => check_fn(&self.table_properties), } }

scottsand-db · 2026-04-29T21:45:04Z


+    /// Whether Iceberg compatibility V3 is enabled for this table. When enabled, Delta Lake
+    /// ensures compatibility with Apache Iceberg V3 table format.
+    pub enable_iceberg_compat_v3: Option<bool>,


nit: should these table props really be iceberg_compat_v3_enabled ? this is us asking questions about the existing table properties at a given snapshot, right? just food for thought for future clarity ..

Took some thoughts. Currently similar properties use the pattern(e.g. enable_row_tracking) and feel like we can follow the existing pattern here. The property itself means "hey, please enable iceberg_compat_v3", but yes most cases we are querying them. Perhaps we can expose helpers like iceberg_compat_v3_enabled()->bool in future? E.g. for column mapping

/// Determine the column mapping mode for a table based on the [`Protocol`] and [`TableProperties`] pub(crate) fn column_mapping_mode( protocol: &Protocol, table_properties: &TableProperties, ) -> ColumnMappingMode { match ( table_properties.column_mapping_mode, protocol.min_reader_version(), ) { // NOTE: The table property is optional even when the feature is supported, and is allowed // (but should be ignored) even when the feature is not supported. For details see // https://github.com/delta-io/delta/blob/master/PROTOCOL.md#column-mapping (Some(mode), 2) => mode, (Some(mode), 3) if protocol.has_table_feature(&TableFeature::ColumnMapping) => mode, _ => ColumnMappingMode::None, } }

Or maybe the TableProperties could keep the fields private and expose pub getters?

Jameson-Crate

lgtm

…lta-io#2504) ## Stacked PR Use this [link](https://github.com/delta-io/delta-kernel-rs/pull/2504/files) to review incremental changes. - [stack/add-v3-feature](delta-io#2475) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2475/files)] [MERGED] - [**stack/write-part-when-ice3**](delta-io#2504) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2504/files)] - [stack/block-alter-table](delta-io#2505) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2505/files/15d151f8b271a72cb421ccfa81cda7d5887d1b37..562605f3911acd99854efae7940deb9344e7ee55)] - [stack/write-nested-id](delta-io#2508) [[Files changed](https://github.com/delta-io/delta-kernel-rs/pull/2508/files/562605f3911acd99854efae7940deb9344e7ee55..766ace41c2176c546f533be4ff0d7a9aa24ae001)] --------- ## What changes are proposed in this pull request? Materialize partition values when icebergCompatV3 enabled. ## How was this change tested? 1. When `icebergCompatV3` enabled, partition columns are contained into `physical_write_schema` 2. When `icebergCompatV3` supported but not enabled, partition columns are not contained into `physical_write_schema` Note: other related cases(e.g. `MaterializePartitionColumns`) are already covered by existing tests so we are not adding tests here

github-actions Bot assigned dengsh12 Apr 27, 2026

dengsh12 changed the title ~~first draft~~ feat: Add feature icebergCompatV3 and it's protocol level validation rules Apr 27, 2026

dengsh12 marked this pull request as draft April 27, 2026 23:29

dengsh12 changed the title ~~feat: Add feature icebergCompatV3 and it's protocol level validation rules~~ feat: Add icebergCompatV3 to TableFeature Apr 27, 2026

github-actions Bot added the breaking-change Public API change that could cause downstream compilation failures. Requires a major version bump. label Apr 27, 2026

dengsh12 changed the title ~~feat: Add icebergCompatV3 to TableFeature~~ feat!: Add icebergCompatV3 to TableFeature Apr 28, 2026

dengsh12 requested review from Jameson-Crate and scottsand-db April 28, 2026 00:08

dengsh12 changed the title ~~feat!: Add icebergCompatV3 to TableFeature~~ feat!: Add feature icebergCompatV3 and it's FeatureInfo Apr 28, 2026

dengsh12 marked this pull request as ready for review April 28, 2026 00:18

scottsand-db requested changes Apr 28, 2026

View reviewed changes

dengsh12 mentioned this pull request Apr 28, 2026

Tracking: IcebergCompatV3 support #2492

Open

dengsh12 commented Apr 28, 2026

View reviewed changes

dengsh12 requested a review from scottsand-db April 28, 2026 22:13

scottsand-db approved these changes Apr 29, 2026

View reviewed changes

dengsh12 added 6 commits April 29, 2026 22:27

first draft

a117332

draft

3d853c7

ai reviewed

8d53919

self-review

cb1b612

self-review-2

78bb2ea

fix comments

ba07ed4

Jameson-Crate approved these changes Apr 30, 2026

View reviewed changes

nits

e1a4b55

dengsh12 force-pushed the stack/add-v3-feature branch from dff52db to e1a4b55 Compare April 30, 2026 18:57

This was referenced Apr 30, 2026

feat: Materialize partition values when icebergCompatV3 enabled #2504

Merged

feat: Block alter table when icebergCompatV3 enabled #2505

Open

dengsh12 added this pull request to the merge queue Apr 30, 2026

Merged via the queue into delta-io:main with commit f037ebf Apr 30, 2026
30 of 37 checks passed

dengsh12 deleted the stack/add-v3-feature branch April 30, 2026 23:14

This was referenced May 1, 2026

feat: Write parquet id for nested items in map/array according to metadata #2508

Open

feat: Validate add.stats. numRecords must exist on writes when icebergCompatV3 enabled #2512

Open

Conversation

dengsh12 commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥞 Stacked PR

Stacked PR

What changes are proposed in this pull request?

How was this change tested?

Uh oh!

codecov Bot commented Apr 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

scottsand-db left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dengsh12 Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dengsh12 Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scottsand-db Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dengsh12 Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

scottsand-db left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dengsh12 Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dengsh12 Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jameson-Crate left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

dengsh12 commented Apr 27, 2026 •

edited

Loading

codecov Bot commented Apr 27, 2026 •

edited

Loading

dengsh12 Apr 28, 2026 •

edited

Loading

dengsh12 Apr 28, 2026 •

edited

Loading

scottsand-db Apr 29, 2026 •

edited

Loading

dengsh12 Apr 28, 2026 •

edited

Loading

dengsh12 Apr 30, 2026 •

edited

Loading

dengsh12 Apr 30, 2026 •

edited

Loading