Skip to content

Conversation

@kerwin-zk
Copy link
Contributor

Purpose

Support AlwaysTrue and AlwaysFalse predicates in SparkV2FilterConverter

Tests

CI

@kerwin-zk kerwin-zk force-pushed the always-true-false branch 4 times, most recently from 76f6a9a to 0574ad6 Compare February 6, 2026 04:23
@kerwin-zk
Copy link
Contributor Author

cc @JingsongLi

@JingsongLi
Copy link
Contributor

Why do we need to support this? Are there any beneficial queries?

@JingsongLi
Copy link
Contributor

If we really need to introduce this, I prefer introducing TrueTransform and TrueFunction.

@kerwin-zk
Copy link
Contributor Author

kerwin-zk commented Feb 9, 2026

Why do we need to support this? Are there any beneficial queries?

@JingsongLi

DELETE FROM t1 WHERE pt IN (SELECT id FROM t2 WHERE n > 10) OR pt = 3
The subquery (n > 10) returns empty, evalSubquery replaces it with Literal(false). After translateFilterV2, this becomes: V2Or(ALWAYS_FALSE, V2EqualTo(pt, 3)). Without supporting ALWAYS_FALSE in SparkV2FilterConverter, the OR conversion would fail, falling back to the expensive row-level delete instead of partition dropping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants