Skip to content

Comments

BigQuery: refactor header handling#3829

Merged
ilidemi merged 3 commits intomainfrom
header-provider-main
Jan 12, 2026
Merged

BigQuery: refactor header handling#3829
ilidemi merged 3 commits intomainfrom
header-provider-main

Conversation

@ilidemi
Copy link
Contributor

@ilidemi ilidemi commented Jan 8, 2026

Have one header provider that always returns a fresh token instead of keeping them per-object and mutating in place
Unblocks bringing the buffered channel back

Closes #3828, a follow up to #3782

@ilidemi ilidemi requested a review from jkaflik January 8, 2026 10:00
@codecov
Copy link

codecov bot commented Jan 8, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
1597 2 1595 144
View the top 2 failed test(s) by shortest run time
github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG
Stack Traces | 0.01s run time
=== RUN   TestPeerFlowE2ETestSuitePG
=== PAUSE TestPeerFlowE2ETestSuitePG
=== CONT  TestPeerFlowE2ETestSuitePG
--- FAIL: TestPeerFlowE2ETestSuitePG (0.01s)
2026/01/08 10:09:53 INFO fetched schema x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} table=e2e_test_mychcl_s6kmbdmn.test_extra_ch_cols
github.com/PeerDB-io/peerdb/flow/e2e::TestPeerFlowE2ETestSuitePG/Test_Dynamic_Mirror_Config_Via_Signals
Stack Traces | 79.2s run time
=== RUN   TestPeerFlowE2ETestSuitePG/Test_Dynamic_Mirror_Config_Via_Signals
=== PAUSE TestPeerFlowE2ETestSuitePG/Test_Dynamic_Mirror_Config_Via_Signals
=== CONT  TestPeerFlowE2ETestSuitePG/Test_Dynamic_Mirror_Config_Via_Signals
    postgres_test.go:922: Inserted 18 rows into the source table
2026/01/08 10:08:28 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id"
2026/01/08 10:08:28 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id"
2026/01/08 10:08:28 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_8692905791343598008 CURSOR FOR SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id" args=[]
2026/01/08 10:08:28 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id" channelLen=0
2026/01/08 10:08:28 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8692905791343598008
2026/01/08 10:08:28 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8692905791343598008 records=8 bytes=168 channelLen=7
2026/01/08 10:08:28 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id" rows=8 bytes=168 channelLen=0
2026/01/08 10:08:28 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8692905791343598008
2026/01/08 10:08:28 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_8692905791343598008 records=0 bytes=0 channelLen=0
2026/01/08 10:08:28 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/01/08 10:08:28 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/01/08 10:08:28 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,name,created_at FROM e2e_test_bq_xqaapbcr_20260108100732.\"test_inheritance_dynconf\" ORDER BY id" rows=8 bytes=168 channelLen=0
    postgres_test.go:922: Inserted 18 rows into the source table
    postgres_test.go:931: WaitFor normalize 18 records - first table 2026-01-08 10:08:32.205161181 +0000 UTC m=+218.099067980
    postgres_test.go:931: UNEXPECTED TIMEOUT normalize 18 records - first table 2026-01-08 10:09:32.299196529 +0000 UTC m=+278.193103325
    postgres.go:92: begin tearing down postgres schema pg_z13vfppu
--- FAIL: TestPeerFlowE2ETestSuitePG/Test_Dynamic_Mirror_Config_Via_Signals (79.23s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@ilidemi ilidemi requested a review from jgao54 January 8, 2026 19:08
Copy link
Contributor

@jkaflik jkaflik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool change @ilidemi

@ilidemi ilidemi merged commit 54d27d1 into main Jan 12, 2026
14 of 16 checks passed
@ilidemi ilidemi deleted the header-provider-main branch January 12, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use header provider callback in QRepObjectSync

2 participants