fix: allow non -chunked streams w/o Content-Length header

[stefan-tb]

Plain (non chunked) HTTP bodies from server responses without a Content-Length header can't be read at client side.

Example:

auto stream = cli.open_stream("GET", "/stream-path");
auto n = stream.read(buffer, sizeof(buffer)); // always returns 0

That's because in the case of the missing header, the method ClientImpl::open_stream doesn't set BodyReader::content_length and the value remains entirely untouched (default value 0):

cpp-httplib/httplib.h

Lines 10448 to 10453 in fbec2a3

	auto content_length_str = handle.response->get_header_value("Content-Length");
	if (!content_length_str.empty()) {
	handle.body_reader_.content_length =
	static_cast<size_t>(std::stoull(content_length_str));
	}

Therefore consecutive tests in BodyReader::read, like this one:

cpp-httplib/httplib.h

Lines 8128 to 8131 in fbec2a3

	if (bytes_read >= content_length) {
	eof = true;
	return 0;
	}

will immediately set eof and return 0 before even trying to read any actual data.

This PR is introducing another field bool has_content_length (std::optional is not C++11 as far as I know) to track if BodyReader::content_length has been set. If not, the related tests are skipped. Please note that the eol flag will still be properly set when a read from the underlying stream returns 0.

I am new to this code base, this is a best effort commit. I tried to keep things as local as possible. Sorry in advance if I am violating style / approaches.

[yhirose]

@stefan-tb Thanks for the pull request. Could you please add the unit test in test/test.cc that I always ask contributors to include? Thanks!

[stefan-tb]

@yhirose: wow. 🥇 for your response time. Honestly.
I will. I need to look into them. Might take a week.

Thanks

[yhirose]

@stefan-tb I just had the Copilot review this pull request, and here is the summary. Could you take a look at this potential problem, too?

This PR introduces a potential denial-of-service (DoS) vulnerability on the client side. When an HTTP response lacks both the Content-Length header and Transfer-Encoding: chunked, the client will now read the body until the connection is closed, without any built-in limit. As a result, if a malicious server sends an unbounded or extremely large body, the client may consume excessive memory or disk resources and cannot protect itself by default. Unlike the server side, which enforces a maximum payload size (set_payload_max_length), there is no similar safeguard for client responses, making this change dangerous without additional resource limits.

I have already fixed the same issue on the server side before. You can see the code in the following locations.

Payload Max Length ExceedLimit:

cpp-httplib/httplib.h

Lines 9126 to 9131 in fbec2a3

	// There is data, so read it with payload limit enforcement
	auto result = detail::read_content_without_length(
	strm, payload_max_length_, out);
	if (result == detail::ReadContentResult::PayloadTooLarge) {
	res.status = StatusCode::PayloadTooLarge_413;
	return false;

cpp-httplib/test/test.cc

Lines 8244 to 8507 in fbec2a3

	TEST_F(PayloadMaxLengthTest, ExceedLimit) {
	auto res = cli_.Post("/test", "123456789", "text/plain");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::PayloadTooLarge_413, res->status);
	res = cli_.Post("/test", "12345678", "text/plain");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::OK_200, res->status);
	}
	TEST_F(PayloadMaxLengthTest, ChunkedEncodingSecurityTest) {
	// Test chunked encoding with payload exceeding the 8-byte limit
	std::string large_chunked_data(16, 'A'); // 16 bytes, exceeds 8-byte limit
	auto res = cli_.Post("/test", large_chunked_data, "text/plain");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::PayloadTooLarge_413, res->status);
	}
	TEST_F(PayloadMaxLengthTest, ChunkedEncodingWithinLimit) {
	// Test chunked encoding with payload within the 8-byte limit
	std::string small_chunked_data(4, 'B'); // 4 bytes, within 8-byte limit
	auto res = cli_.Post("/test", small_chunked_data, "text/plain");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::OK_200, res->status);
	}
	TEST_F(PayloadMaxLengthTest, RawSocketChunkedTest) {
	// Test using send_request to send chunked data exceeding payload limit
	std::string chunked_request = "POST /test HTTP/1.1\r\n"
	"Host: " +
	std::string(HOST) + ":" + std::to_string(PORT) +
	"\r\n"
	"Transfer-Encoding: chunked\r\n"
	"Connection: close\r\n"
	"\r\n"
	"a\r\n" // 10 bytes chunk (exceeds 8-byte limit)
	"0123456789\r\n"
	"0\r\n" // End chunk
	"\r\n";
	std::string response;
	bool result = send_request(1, chunked_request, &response);
	if (!result) {
	// If send_request fails, it might be because the server closed the
	// connection due to payload limit enforcement, which is acceptable
	SUCCEED()
	<< "Server rejected oversized chunked request (connection closed)";
	} else {
	// If we got a response, check if it's an error response or connection was
	// closed early Short response length indicates connection was closed due to
	// payload limit
	if (response.length() <= 10) {
	SUCCEED() << "Server closed connection for oversized chunked request";
	} else {
	// Check for error status codes
	EXPECT_TRUE(response.find("413") != std::string::npos ||
	response.find("Payload Too Large") != std::string::npos ||
	response.find("400") != std::string::npos);
	}
	}
	}
	TEST_F(PayloadMaxLengthTest, NoContentLengthPayloadLimit) {
	// Test request without Content-Length header exceeding payload limit
	std::string request_without_content_length = "POST /test HTTP/1.1\r\n"
	"Host: " +
	std::string(HOST) + ":" +
	std::to_string(PORT) +
	"\r\n"
	"Connection: close\r\n"
	"\r\n";
	// Add payload exceeding the 8-byte limit
	std::string large_payload(16, 'X'); // 16 bytes, exceeds 8-byte limit
	request_without_content_length += large_payload;
	std::string response;
	bool result = send_request(1, request_without_content_length, &response);
	if (!result) {
	// If send_request fails, server likely closed connection due to payload
	// limit
	SUCCEED() << "Server rejected oversized request without Content-Length "
	"(connection closed)";
	} else {
	// Check if server responded with error or closed connection early
	if (response.length() <= 10) {
	SUCCEED() << "Server closed connection for oversized request without "
	"Content-Length";
	} else {
	// Check for error status codes
	EXPECT_TRUE(response.find("413") != std::string::npos ||
	response.find("Payload Too Large") != std::string::npos ||
	response.find("400") != std::string::npos);
	}
	}
	}
	TEST_F(PayloadMaxLengthTest, NoContentLengthWithinLimit) {
	// Test request without Content-Length header within payload limit
	std::string request_without_content_length = "POST /test HTTP/1.1\r\n"
	"Host: " +
	std::string(HOST) + ":" +
	std::to_string(PORT) +
	"\r\n"
	"Connection: close\r\n"
	"\r\n";
	// Add payload within the 8-byte limit
	std::string small_payload(4, 'Y'); // 4 bytes, within 8-byte limit
	request_without_content_length += small_payload;
	std::string response;
	bool result = send_request(1, request_without_content_length, &response);
	// For requests without Content-Length, the server may have different behavior
	// The key is that it should not reject due to payload limit for small
	// payloads
	if (result) {
	// Check for any HTTP response (success or error, but not connection closed)
	if (response.length() > 10) {
	SUCCEED()
	<< "Server processed request without Content-Length within limit";
	} else {
	// Short response might indicate connection closed, which is acceptable
	SUCCEED() << "Server closed connection for request without "
	"Content-Length (acceptable behavior)";
	}
	} else {
	// Connection failure might be due to protocol requirements
	SUCCEED() << "Connection issue with request without Content-Length "
	"(environment-specific)";
	}
	}
	class LargePayloadMaxLengthTest : public ::testing::Test {
	protected:
	LargePayloadMaxLengthTest()
	: cli_(HOST, PORT)
	#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
	,
	svr_(SERVER_CERT_FILE, SERVER_PRIVATE_KEY_FILE)
	#endif
	{
	#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
	cli_.enable_server_certificate_verification(false);
	#endif
	}
	virtual void SetUp() {
	// Set 10MB payload limit
	const size_t LARGE_PAYLOAD_LIMIT = 10 * 1024 * 1024; // 10MB
	svr_.set_payload_max_length(LARGE_PAYLOAD_LIMIT);
	svr_.Post("/test", [&](const Request & /*req*/, Response &res) {
	res.set_content("Large payload test", "text/plain");
	});
	t_ = thread([&]() { ASSERT_TRUE(svr_.listen(HOST, PORT)); });
	svr_.wait_until_ready();
	}
	virtual void TearDown() {
	svr_.stop();
	t_.join();
	}
	#ifdef CPPHTTPLIB_OPENSSL_SUPPORT
	SSLClient cli_;
	SSLServer svr_;
	#else
	Client cli_;
	Server svr_;
	#endif
	thread t_;
	};
	TEST_F(LargePayloadMaxLengthTest, ChunkedEncodingWithin10MB) {
	// Test chunked encoding with payload within 10MB limit
	std::string medium_payload(5 * 1024 * 1024,
	'A'); // 5MB payload, within 10MB limit
	auto res = cli_.Post("/test", medium_payload, "application/octet-stream");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::OK_200, res->status);
	}
	TEST_F(LargePayloadMaxLengthTest, ChunkedEncodingExceeds10MB) {
	// Test chunked encoding with payload exceeding 10MB limit
	std::string large_payload(12 * 1024 * 1024,
	'B'); // 12MB payload, exceeds 10MB limit
	auto res = cli_.Post("/test", large_payload, "application/octet-stream");
	ASSERT_TRUE(res);
	EXPECT_EQ(StatusCode::PayloadTooLarge_413, res->status);
	}
	TEST_F(LargePayloadMaxLengthTest, NoContentLengthWithin10MB) {
	// Test request without Content-Length header within 10MB limit
	std::string request_without_content_length = "POST /test HTTP/1.1\r\n"
	"Host: " +
	std::string(HOST) + ":" +
	std::to_string(PORT) +
	"\r\n"
	"Connection: close\r\n"
	"\r\n";
	// Add 1MB payload (within 10MB limit)
	std::string medium_payload(1024 * 1024, 'C'); // 1MB payload
	request_without_content_length += medium_payload;
	std::string response;
	bool result = send_request(5, request_without_content_length, &response);
	if (result) {
	// Should get a proper HTTP response for payloads within limit
	if (response.length() > 10) {
	SUCCEED() << "Server processed 1MB request without Content-Length within "
	"10MB limit";
	} else {
	SUCCEED() << "Server closed connection (acceptable behavior for no "
	"Content-Length)";
	}
	} else {
	SUCCEED() << "Connection issue with 1MB payload (environment-specific)";
	}
	}
	TEST_F(LargePayloadMaxLengthTest, NoContentLengthExceeds10MB) {
	// Test request without Content-Length header exceeding 10MB limit
	std::string request_without_content_length = "POST /test HTTP/1.1\r\n"
	"Host: " +
	std::string(HOST) + ":" +
	std::to_string(PORT) +
	"\r\n"
	"Connection: close\r\n"
	"\r\n";
	// Add 12MB payload (exceeds 10MB limit)
	std::string large_payload(12 * 1024 * 1024, 'D'); // 12MB payload
	request_without_content_length += large_payload;
	std::string response;
	bool result = send_request(10, request_without_content_length, &response);
	if (!result) {
	// Server should close connection due to payload limit
	SUCCEED() << "Server rejected 12MB request without Content-Length "
	"(connection closed)";
	} else {
	// Check for error response
	if (response.length() <= 10) {
	SUCCEED()
	<< "Server closed connection for 12MB request exceeding 10MB limit";
	} else {
	EXPECT_TRUE(response.find("413") != std::string::npos ||
	response.find("Payload Too Large") != std::string::npos ||
	response.find("400") != std::string::npos);
	}
	}
	}

Decompressed Size Exceeds Limit:

cpp-httplib/httplib.h

Lines 9016 to 9023 in fbec2a3

	// Limit decompressed body size to payload_max_length_ to protect
	// against "zip bomb" attacks where a small compressed payload
	// decompresses to a massive size.
	if (payload_max_length_ > 0 &&
	(req.body.size() >= payload_max_length_ ||
	n > payload_max_length_ - req.body.size())) {
	return false;
	}

cpp-httplib/test/test.cc

Line 14240 in fbec2a3

TEST(ZipBombProtectionTest, DecompressedSizeExceedsLimit) {

Thanks!

[stefan-tb]

My changes only apply to non-chunked, client-side streaming scenarios. Calling cli.Get("/") is doing its entirely own thing (to my understanding from Debugging my entire way my thru it) and reads "only" res.body.max_size() in case of a missing header, which is effectively int64.MaxValue and therefore quite large.