fix: respect -pr http11 flag by disabling HTTP/2 fallback

[285729101]

Proposed Changes

/claim #2240

Fixes #2240

Problem

When running httpx with -pr http11, the transport is correctly configured to disable HTTP/2:

if httpx.Options.Protocol == "http11" {
    os.Setenv("GODEBUG", "http2client=0")
    transport.TLSNextProto = map[string]func(string, *tls.Conn) http.RoundTripper{}
}

However, retryablehttp-go creates a separate HTTPClient2 with HTTP/2 enabled. In do.go, when certain protocol errors are encountered, it falls back to HTTPClient2:

if err != nil && stringsutil.ContainsAny(err.Error(), 
    "net/http: HTTP/1.x transport connection broken: malformed HTTP version \"HTTP/2\"",
    "net/http: HTTP/1.x transport connection broken: malformed HTTP response") {
    resp, err = c.HTTPClient2.Do(req.Request)
}

This bypasses the user's explicit HTTP/1.1 protocol selection, making -pr http11 unreliable.

Solution

After creating the retryablehttp client, when Protocol == "http11", set HTTPClient2 to the same client as HTTPClient. This prevents retryablehttp's fallback from switching to an HTTP/2-capable client:

if httpx.Options.Protocol == "http11" {
    httpx.client.HTTPClient2 = httpx.client.HTTPClient
}

This approach:

• No external dependency changes - works entirely within httpx, no retryablehttp-go modifications needed
• Backward compatible - default behavior unchanged; HTTP/2 fallback still works when no explicit protocol is set
• Minimal and focused - single line of logic that directly addresses the root cause

Proof

All tests pass locally (fully offline, no external network calls):

$ go test ./common/httpx/ -count=1 -v -run "TestHTTP11|TestDefault|TestDo"
=== RUN   TestDo
=== RUN   TestDo/content-length_in_header
--- PASS: TestDo (0.09s)
    --- PASS: TestDo/content-length_in_header (0.00s)
=== RUN   TestHTTP11DisablesHTTP2Fallback
--- PASS: TestHTTP11DisablesHTTP2Fallback (0.05s)
=== RUN   TestDefaultProtocolKeepsHTTP2Fallback
--- PASS: TestDefaultProtocolKeepsHTTP2Fallback (0.06s)
PASS
ok      github.com/projectdiscovery/httpx/common/httpx  0.293s

Tests Added

• TestHTTP11DisablesHTTP2Fallback - verifies HTTPClient2 == HTTPClient (pointer identity) when -pr http11 is set
• TestDefaultProtocolKeepsHTTP2Fallback - verifies HTTPClient2 != HTTPClient in default mode (HTTP/2 fallback preserved)
• Updated TestDo to use httptest.NewServer instead of external URLs for offline testing

Checklist

• PR created against the correct branch (dev)
• All checks passed (lint, unit/integration/regression tests)
• Tests added that prove the fix is effective
• Documentation added (inline comments explaining the fix)

Summary by CodeRabbit

• Bug Fixes

  • Fixed protocol handling to ensure HTTP/1.1 connections remain on HTTP/1.1 and are not automatically switched to HTTP/2 during operation.

• Tests

  • Added comprehensive test coverage for protocol-specific connection behavior and HTTP/2 fallback scenarios.

[coderabbitai]

Walkthrough

This change fixes the HTTP/1.1 protocol flag being ignored due to HTTP/2 fallback in retryablehttp-go. When protocol is set to "http11", the HTTP/2 fallback client is reassigned to use the standard HTTP client, preventing unintended protocol switches on connection errors.

Changes

Cohort / File(s)	Summary
HTTP/1.1 Client Selection Logic common/httpx/httpx.go	Adds conditional that assigns HTTPClient2 to HTTPClient when Protocol is "http11", preventing retryablehttp-go from falling back to HTTP/2 on malformed responses.
Protocol Fallback Tests common/httpx/httpx_test.go	Replaces remote favicon fetch test with local httptest server. Adds TestHTTP11DisablesHTTP2Fallback to verify client equality when http11 is set and TestDefaultProtocolKeepsHTTP2Fallback to verify distinct clients by default.

Estimated Code Review Effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 A flag once ignored, now takes its stand,
When http11 is passed, we understand—
No sneaky switch to version two,
The protocol path stays tried and true! ✨

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately summarizes the main change: disabling HTTP/2 fallback to respect the -pr http11 flag when explicitly set.
Linked Issues check	✅ Passed	The code changes fully address issue #2240 by setting HTTPClient2=HTTPClient when Protocol is http11, preventing HTTP/2 fallback on protocol errors.
Out of Scope Changes check	✅ Passed	All changes are scoped to the HTTP/1.1 protocol handling objective; no unrelated modifications are present.
Merge Conflict Detection	✅ Passed	✅ No merge conflicts detected when merging into dev

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

common/httpx/httpx_test.go (1)

51-58: Minor: &DefaultOptions is passed directly, which New() may mutate.

New() calls parseCustomCookies() on the options pointer (line 74 of httpx.go). Passing &DefaultOptions directly could mutate the package-level variable, potentially causing test interference depending on execution order. Consider copying like the http11 test does:

-	ht, err := New(&DefaultOptions)
+	opts := DefaultOptions
+	ht, err := New(&opts)

The same pattern exists in TestDo (line 13), but since you're already doing it correctly in TestHTTP11DisablesHTTP2Fallback, it'd be good to be consistent.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@common/httpx/httpx_test.go` around lines 51 - 58,
TestDefaultProtocolKeepsHTTP2Fallback passes &DefaultOptions directly which
New() (via parseCustomCookies) may mutate; change the test to pass a local copy
of DefaultOptions (e.g., create a local variable like opt := DefaultOptions and
call New(&opt)) so New() mutates the copy not the package-level
DefaultOptions—follow the same pattern used in TestHTTP11DisablesHTTP2Fallback
and TestDo.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@common/httpx/httpx_test.go`:
- Around line 51-58: TestDefaultProtocolKeepsHTTP2Fallback passes
&DefaultOptions directly which New() (via parseCustomCookies) may mutate; change
the test to pass a local copy of DefaultOptions (e.g., create a local variable
like opt := DefaultOptions and call New(&opt)) so New() mutates the copy not the
package-level DefaultOptions—follow the same pattern used in
TestHTTP11DisablesHTTP2Fallback and TestDo.