fix(metadata): replace ReadTimeout with ReadHeaderTimeout on metadata server#38
Closed
prosdev wants to merge 1 commit intoantflydb:mainfrom
Closed
fix(metadata): replace ReadTimeout with ReadHeaderTimeout on metadata server#38prosdev wants to merge 1 commit intoantflydb:mainfrom
prosdev wants to merge 1 commit intoantflydb:mainfrom
Conversation
ajroetker
reviewed
Apr 2, 2026
ajroetker
reviewed
Apr 2, 2026
|
|
||
| // TestLinearMerge_KeySortingLargeScale verifies key extraction and sorting | ||
| // at scale (5,000 records), matching the handler's sort behavior. | ||
| func TestLinearMerge_KeySortingLargeScale(t *testing.T) { |
Contributor
There was a problem hiding this comment.
Similarly does this fail without the patch?
Author
There was a problem hiding this comment.
Validation was E2E: cli/cli (5,933 records) fails with ReadTimeout=10s, works with ReadHeaderTimeout=30s.
… server ReadTimeout=10s applied to the entire HTTP request (headers + body), causing JSON decode failures on large Linear Merge payloads. With ~6k records (~7MB), the server closed the connection mid-read, producing: decoding request: json: string unexpected end of JSON input ReadHeaderTimeout=30s limits only header reading, allowing large request bodies to complete. This matches the pattern used by the Raft server (30s) and Health server (40s) in this codebase. Tested: cli/cli (830 Go files, 5,933 components) now indexes successfully via dev-agent where it previously failed. Fixes antflydb#37
prosdev
force-pushed
the
fix/metadata-read-header-timeout
branch
from
3daa34a to
bc04ecf
Compare
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace
ReadTimeout: 10 * time.SecondwithReadHeaderTimeout: 30 * time.Secondon the metadata HTTP server. This fixes large Linear Merge payloads failing with JSON decode errors.Fixes #37
Problem
The metadata server's
ReadTimeoutapplies to the entire HTTP request (headers + body). Large Linear Merge payloads (~7MB for 6,000 records) can't be fully transmitted within 10 seconds, causing the server to close the connection mid-read:Fix
ReadHeaderTimeoutlimits only header reading (protection against slowloris attacks) while allowing request bodies to take as long as needed. This matches the pattern already used by:ReadHeaderTimeout: 30 * time.Second(src/raft/multiraft.go:268)ReadHeaderTimeout: 40 * time.Second(pkg/libaf/healthserver/healthserver.go:63)Linear Merge still enforces
MaxRecordsPerRequest = 10,000— no size protection is removed.Changes
src/metadata/runner.go:76ReadTimeout: 10s→ReadHeaderTimeout: 30se2e/swarm.go:289src/metadata/api_linear_merge_test.goTesting