Support for executing DistSQL for ShardingSphere JDBC logical databases

[linghengqian]

For #37761 .

Changes proposed in this pull request:

• Support for executing DistSQL for ShardingSphere JDBC logical databases.
  • Since some infrastructure exists only within ShardingSphere Proxy, I copied org.apache.shardingsphere.proxy.backend.distsql.DistSQLStatementContext to org.apache.shardingsphere.driver.jdbc.core.statement.DistSQLStatementContext.
• The current PR needs to be processed after being merged at Fix failing nativeTest on Windows Server 2025 #37775 .
• The current PR does not discuss refactoring the org.apache.shardingsphere:shardingsphere-test-native module, nor does it discuss implementing support for JDBC URLs like jdbc:shardingsphere:scratch:logic_db that lack a YAML file style. This will be addressed through some separate issues.
• Dynamically modifying the databaseName and props properties via DistSQL will not be addressed in the current PR.
• The current PR will not implement support for using CREATE DATABASE sharding_db or USE sharding_db on ShardingSphere JDBC.
• Because the unit tests for the JDBC and the Proxy in org.apache.shardingsphere:shardingsphere-test-native are not yet separated into different modules, the documentation may be missing some Maven modules. I haven't had time to check [ERROR] Metadata copy process failed with code: 1 is thrown when using native:metadata-copy in a multi-module Maven project graalvm/native-build-tools#650 and Native Build Tools 0.9.26's native:metadata-copy causes MojoExecutionException when executed on a multi-module Maven project graalvm/native-build-tools#500 yet, since I'm obviously not a graalvm committer.

---

Before committing this PR, I'm sure that I have checked the following options:

• My code follows the code of conduct of this project.
• I have self-reviewed the commit code.
• I have (or in comment I request) added corresponding labels for the pull request.
• I have passed maven check locally : ./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e.
• I have made corresponding changes to the documentation.
• I have added corresponding unit tests for my changes.
• I have updated the Release Notes of the current development version. For more details, see Update Release Note

[terrymanu]

Good direction: wiring DistSQL into JDBC with a dedicated executor/result set and adding JDBC-side DistSQL docs is the right direction—please keep pushing on this path.

Change requests:

• Most DistSQL executors live under proxy/backend/handler/distsql, and the JDBC module only depends on infra-distsql-handler. In JDBC the majority of DistSQL statements will not find executors on the classpath and will fail. Please either provide JDBC-usable executors for the intended statements or clearly scope the supported DistSQL set (with tests/docs).
• DriverDistSQLExecutor#createDistSQLConnectionContext builds a context with JDBC’s DatabaseConnectionManager and a null executorStatementManager, while executors like proxy/backend/core/.../PreviewExecutor.java:137 expect a ProxyDatabaseConnectionManager and a non-null statement manager. In JDBC this will lead to ClassCastException/NullPointerException. Please supply a JDBC-compatible context or adjust executors to be JDBC-safe.
• ShardingSphereStatement caches metaData at construction; after DistSQL updates, the same statement continues to use stale metadata, so newly registered storage units/rules are invisible to subsequent SQL. Please refresh metadata per execution or otherwise sync with MetaDataContexts.
• test/native/.../ShardingTest now depends on DistSQL dynamic registration; given the above gaps, this path will fail. Consider keeping the YAML fallback or delay the test switch until DistSQL-on-JDBC is functional.

Please address the above and add integration coverage for “DistSQL register/create → regular DML” in JDBC once the fixes are in. Keep going—this feature will be valuable once the JDBC path is complete.

[linghengqian]

DriverDistSQLExecutor#createDistSQLConnectionContext builds a context with JDBC’s DatabaseConnectionManager and a null executorStatementManager, while executors like proxy/backend/core/.../PreviewExecutor.java:137 expect a ProxyDatabaseConnectionManager and a non-null statement manager. In JDBC this will lead to ClassCastException/NullPointerException. Please supply a JDBC-compatible context or adjust executors to be JDBC-safe.

I don't see any unit tests throwing ClassCastException locally or in CI. Where is this error coming from?

test/native/.../ShardingTest now depends on DistSQL dynamic registration; given the above gaps, this path will fail. Consider keeping the YAML fallback or delay the test switch until DistSQL-on-JDBC is functional.

ShardingTest did not fail. Where are the failure logs? The tests containing this one, https://github.com/apache/shardingsphere/actions/runs/21162151539/job/60861111930 and https://github.com/apache/shardingsphere/actions/runs/21162151539/job/60861112341, both succeeded. Dozens of other unit tests using YAML in the same directory as this unit test are still working correctly.

[terrymanu]

Most DistSQL executors live under proxy/backend/handler/distsql, and the JDBC module only depends on infra-distsql-handler.

This is core issue

[linghengqian]

Most DistSQL executors live under proxy/backend/handler/distsql, and the JDBC module only depends on infra-distsql-handler.

This is core issue

My idea is that there shouldn't be any DistSQL instances that ShardingSphere JDBC can't use because the handling of the ContextManager is shared. If executing DistSQL previously threw an exception in JDBC, it will still throw the same exception if a Maven module becomes unreachable.

It's difficult to test the specific list of available DistSQL instances within JDBC class-level unit tests because ShardingSphere's existing unit tests don't involve the actual database. Once testing the actual database is required, a large number of integration tests containing DistSQL instances would need to be added to test-e2e or test-native, which doesn't seem suitable for the current PR.

[terrymanu]

I accept this requirement, JDBC should be able to use DistSQL just like Proxy.

However, the current situation is that a large portion of DistSQL implementations reside in the Proxy module, so simply enabling JDBC calls won’t achieve that.

It would first require moving the DistSQL-related logic from Proxy up to the kernel module.

[linghengqian]

It would first require moving the DistSQL-related logic from Proxy up to the kernel module.

So, you mean to first merge #37867 , and then make JDBC Core depend on org.apache.shardingsphere:shardingsphere-distsql-handler in this PR, right?

[terrymanu]

This feature is too complex.
Could we please design the modification plan in #37761, split the tasks, and then submit pull requests separately for each task?

[linghengqian]

• I've changed the description of #37761 and the current PR; the current PR will no longer directly close the original issue. The original issue now has a task list.

  • The distsql executor for RAL and RUL statements, which are independent of the ShardingSphere proxy, will be migrated into a kernel submodule.
  • Modify DriverDatabaseConnectionManager and ShardingSphereStatement to support executing distsql on the JDBC Driver.
  • Add E2E for DistSQL and ShardingSphere JDBC.
  • Refactor the shardingsphere-test-native module to avoid most of the YAML file.
  • Add an infra URL impl like jdbc:shardingsphere:scratch:sharding_db to avoid creating the YAML file.
  • Refactor the shardingsphere-test-native module to avoid all YAML file.
  • Add document for DistSQL and ShardingSphere JDBC.

• Does this mean that in the current PR, I only need to delete the newly added document and wait for another PR to merge it?