-
Notifications
You must be signed in to change notification settings - Fork 20
Open
Description
Describe the bug
When connecting to a FlightSQL server and executing queries that return multiple
batches (e.g., GROUP BY without ORDER BY), dft only displays the last batch
received instead of concatenating all batches together.
To Reproduce
Tested against two independent FlightSQL server implementations with identical results:
- Custom server using
arrow-flight53.x + DataFusion 44 datafusion-flight-sql-server0.4.7 + DataFusion 46
Both servers return correct results when queried with a Rust FlightSqlServiceClient
(all batches collected via stream.try_collect()).
Steps:
- Connect to a FlightSQL server:
dft --flightsql-host "http://localhost:50051" - Run an aggregation query:
SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol; - Observe only 1 row is returned
- Run the same query multiple times - the single row changes between executions
(partition completion order is non-deterministic)
Expected behavior
All 3 rows should be displayed (one per group).
Workaround
Adding ORDER BY forces coalescing into a single batch:
SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol ORDER BY symbol;
This correctly returns all 3 rows.
Environment
- dft version: 0.2.3
- Tested against:
- Custom FlightSQL server (arrow-flight 53, DataFusion 44)
- datafusion-flight-sql-server 0.4.7 (DataFusion 46)
Metadata
Metadata
Assignees
Labels
No labels