Use generic error handling for 4XX empty responses#3602
Open
adev-code wants to merge 9 commits intoboto:developfrom
Open
Use generic error handling for 4XX empty responses#3602adev-code wants to merge 9 commits intoboto:developfrom
adev-code wants to merge 9 commits intoboto:developfrom
Conversation
When XML parsing fails due to empty or malformed responses, users now receive the actual HTTP error response from the service alongside the parsing error, providing useful information about what actually happened. Before: ResponseParserError: Unable to parse response (no element found: line 1, column 0), invalid XML received. Further retries may succeed: b'' After: ResponseParserError: Unable to parse response (no element found: line 1, column 0), invalid XML received. Further retries may succeed: b'' HTTP 413: Content Too Large This exposes the real service error (HTTP 413: Content Too Large) that was previously hidden behind cryptic XML parsing failures, giving users actionable information about why their request failed.
adev-code
changed the title
adev-code
changed the title
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #3602 +/- ##
===========================================
- Coverage 93.17% 92.67% -0.50%
===========================================
Files 68 68
Lines 15411 15573 +162
===========================================
+ Hits 14359 14433 +74
- Misses 1052 1140 +88 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The HTTP 413 status message differs between Python versions:
- Python 3.13: "Content Too Large"
- Python 3.11: "Request Entity Too Large"
Updated test assertions to use assertIn("HTTP 413:") instead of exact
string matching to ensure tests pass across supported Python versions
while still verifying that HTTP status context is properly added to
XML parsing errors.
Add hasattr(e, 'add_note') checks before calling add_note() to ensure compatibility with Python 3.9 and 3.10 where PEP 678 exception notes are not available. The enhanced HTTP status context will only be added in Python 3.11+ while maintaining backward compatibility.
nateprewitt
reviewed
Nov 25, 2025
botocore/parsers.py
Outdated
| response['status_code'], '' | ||
| ) | ||
| if status_message: | ||
| if status_message and hasattr(e, 'add_note'): |
Contributor
There was a problem hiding this comment.
I think this requires Python 3.11+ so this is probably not the approach we want to take to make sure this works everywhere.
Contributor
Author
There was a problem hiding this comment.
Thanks Nate, yes I only found out from the CI tests and trying options. Will check for other ways.
Enhanced ResponseParserError messages to include HTTP status codes and messages when XML parsing fails, providing better debugging context for users while maintaining full backward compatibility. Changes: - Modified QueryParser._do_error_parse() to append HTTP status context - Modified RestXMLParser._parse_error_from_body() to append HTTP status context - Enhanced error messages format: "original error (HTTP 413: Content Too Large)" - Preserves original ResponseParserError exception type for compatibility - Added comprehensive tests for both parser classes - Tests handle Python version differences in HTTP status messages
- Improve code readability while maintaining identical functionality
Extend generic error response detection from 5XX to 4XX status codes to prevent XML parsing failures when services return empty bodies. Fixes parsing crashes for cases like AWS Services returning HTTP 413 with empty body, now shows "Content Too Large" instead of "Unable to parse response (no element found: line 1, column 0)".
adev-code
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When AWS services return 4XX errors with empty bodies, users receive XML parsing errors instead of meaningful HTTP status messages. For example, service returns HTTP 413 with an empty body when request size limits are exceeded, but users see:
Solution
This PR extends generic error handling from 5XX to 4XX status codes, preventing parsing failures and surfacing the actual HTTP error, for example:
Backwards Compatibility