Skip to content

FlightSQL client only displays last batch from multi-partition query results #346

@matago

Description

@matago

Describe the bug

When connecting to a FlightSQL server and executing queries that return multiple
batches (e.g., GROUP BY without ORDER BY), dft only displays the last batch
received instead of concatenating all batches together.

To Reproduce

Tested against two independent FlightSQL server implementations with identical results:

  1. Custom server using arrow-flight 53.x + DataFusion 44
  2. datafusion-flight-sql-server 0.4.7 + DataFusion 46

Both servers return correct results when queried with a Rust FlightSqlServiceClient
(all batches collected via stream.try_collect()).

Steps:

  1. Connect to a FlightSQL server:
    dft --flightsql-host "http://localhost:50051"
    
  2. Run an aggregation query:
    SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol;
  3. Observe only 1 row is returned
  4. Run the same query multiple times - the single row changes between executions
    (partition completion order is non-deterministic)

Expected behavior

All 3 rows should be displayed (one per group).

Workaround

Adding ORDER BY forces coalescing into a single batch:
SELECT symbol, AVG(close) as avg_close FROM sample_stocks GROUP BY symbol ORDER BY symbol;
This correctly returns all 3 rows.

Environment

  • dft version: 0.2.3
  • Tested against:
    • Custom FlightSQL server (arrow-flight 53, DataFusion 44)
    • datafusion-flight-sql-server 0.4.7 (DataFusion 46)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions