feat(network-details): Implement header and body extraction#7585
feat(network-details): Implement header and body extraction#758543jay wants to merge 11 commits intomobile-935/new-swizzlingfrom
Conversation
Semver Impact of This PR🟡 Minor (new features) 📋 Changelog PreviewThis is how your changes will appear in the changelog. This PR will not appear in the changelog. 🤖 This preview updates automatically when you update the PR. |
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## mobile-935/new-swizzling #7585 +/- ##
==============================================================
+ Coverage 84.798% 85.007% +0.208%
==============================================================
Files 486 486
Lines 28958 29054 +96
Branches 12564 12618 +54
==============================================================
+ Hits 24556 24698 +142
+ Misses 4353 4308 -45
+ Partials 49 48 -1
... and 4 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
9473efb to
5bc5830
Compare
3547546 to
e7c5abb
Compare
7faf4ef to
6e19c0a
Compare
e7c5abb to
2e9607e
Compare
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Outdated
Show resolved
Hide resolved
6e19c0a to
9a8d48d
Compare
6e5c5bb to
41cf944
Compare
9a8d48d to
8584657
Compare
41cf944 to
68fbe88
Compare
c3afd6d to
6e873ec
Compare
68fbe88 to
6a06365
Compare
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Show resolved
Hide resolved
499493d to
fbd791c
Compare
a691160 to
7db5474
Compare
| let utType = mimeType.flatMap { UTType(mimeType: $0) } | ||
| if let utType, utType.conforms(to: .json) { | ||
| if isTruncated { warnings.append(.jsonTruncated) } | ||
| return parseJSON(data, warnings: &warnings) | ||
| } else if utType?.conforms(to: .text) == true { | ||
| if isTruncated { warnings.append(.textTruncated) } | ||
| return parseText(data, warnings: &warnings) | ||
| } | ||
| return nil |
There was a problem hiding this comment.
l: nitpick, no point in using if-else
| let utType = mimeType.flatMap { UTType(mimeType: $0) } | |
| if let utType, utType.conforms(to: .json) { | |
| if isTruncated { warnings.append(.jsonTruncated) } | |
| return parseJSON(data, warnings: &warnings) | |
| } else if utType?.conforms(to: .text) == true { | |
| if isTruncated { warnings.append(.textTruncated) } | |
| return parseText(data, warnings: &warnings) | |
| } | |
| return nil | |
| guard let utType = mimeType.flatMap { UTType(mimeType: $0) } else { | |
| return nil | |
| } | |
| if utType.conforms(to: .json) { | |
| if isTruncated { warnings.append(.jsonTruncated) } | |
| return parseJSON(data, warnings: &warnings) | |
| } | |
| if utType.conforms(to: .text) { | |
| if isTruncated { warnings.append(.textTruncated) } | |
| return parseText(data, warnings: &warnings) | |
| } | |
| return nil |
|
|
||
| private static func parseFormEncoded(_ data: Data, warnings: inout [NetworkBodyWarning]) -> Body { | ||
| guard let string = String(data: data, encoding: .utf8) ?? String(data: data, encoding: .isoLatin1), | ||
| let components = URLComponents(string: "http://x?" + string), |
There was a problem hiding this comment.
h: I am not convinced that this is the state-of-the-art to parse form-url-encoded data. Is this how other libraries do it? We should instead consider using an implementation of the Decoder protocol e.g. FormUrlDecoder which works similarly to the JSONDecoder.
Also you assume that the body type is either UTF-8 or ISO-Latin-1, but http requests should actually have a content-type header which defines the encoding format.
There was a problem hiding this comment.
not convinced that this is the state-of-the-art to parse form-url-encoded data
Good shout. I looked into this more.
AFAICT, there's no easy soln - either 1) ship a Decoder-conforming FormUrlDecoder that is a bunch of code that needs proper testing for this narrow use-case, or 2) do something simple and incomplete inline.
Deferring to your preference, I propose option 3) Don't support extracting x-www-form-urlencoded as structured data: just extract it as text.
It won't show up as structured data (expand/collapsible) on dashboard but all the data will be there to see + no weird edge cases to confuse ppl.
Some findings:
Apple's own engineer (Quinn "The Eskimo!") explicitly stated in Apple Developer Forums #113632 that x-www-form-urlencoded is "not supported by URLComponents"
From Apple Developer Forums #113632:
Creating data in this format is tricky because the specification is so weak. Below you’ll find my take on it.
Known bugs with URLComponents for form data:
- not decoded as space — in form-encoding, + means space; URLComponents treats it as a literal + (rdar://40751862)
Semicolons not handled as alternate separators
No nested key support (user[name]=foo)
option 1) Decoder-conforming impl // first-pass approximation via claude:
/// Decodes application/x-www-form-urlencoded data into Decodable types.
final class FormUrlDecoder {
func decode<T: Decodable>(_ type: T.Type, from data: Data) throws -> T {
guard let string = String(data: data, encoding: .utf8) else {
throw DecodingError.dataCorrupted(.init(codingPath: [], debugDescription: "Not valid UTF-8"))
}
let pairs = Self.parse(string)
let decoder = _FormUrlDecoder(pairs: pairs)
return try T(from: decoder)
}
/// Splits on `&`, then `=`, decodes `+` as space, percent-decodes.
private static func parse(_ string: String) -> [(String, String)] {
string.split(separator: "&", omittingEmptySubsequences: true).compactMap { pair in
let parts = pair.split(separator: "=", maxSplits: 1)
guard let key = String(parts[0])
.replacingOccurrences(of: "+", with: " ")
.removingPercentEncoding, !key.isEmpty else { return nil }
let value = parts.count > 1
? String(parts[1]).replacingOccurrences(of: "+", with: " ").removingPercentEncoding ?? ""
: ""
return (key, value)
}
}
}
// MARK: - Internal Decoder
private struct _FormUrlDecoder: Decoder {
let pairs: [(String, String)]
var codingPath: [CodingKey] = []
var userInfo: [CodingUserInfoKey: Any] = [:]
func container<Key: CodingKey>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
KeyedDecodingContainer(_KeyedContainer<Key>(pairs: pairs, codingPath: codingPath))
}
func unkeyedContainer() throws -> UnkeyedDecodingContainer {
throw DecodingError.typeMismatch([Any].self,
.init(codingPath: codingPath, debugDescription: "Form data does not support unkeyed containers"))
}
func singleValueContainer() throws -> SingleValueDecodingContainer {
throw DecodingError.typeMismatch(Any.self,
.init(codingPath: codingPath, debugDescription: "Form data does not support single value containers"))
}
}
// MARK: - KeyedDecodingContainer
private struct _KeyedContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
let pairs: [(String, String)]
var codingPath: [CodingKey]
var allKeys: [Key] { pairs.compactMap { Key(stringValue: $0.0) } }
func contains(_ key: Key) -> Bool {
pairs.contains { $0.0 == key.stringValue }
}
private func value(for key: Key) throws -> String {
guard let pair = pairs.first(where: { $0.0 == key.stringValue }) else {
throw DecodingError.keyNotFound(key,
.init(codingPath: codingPath, debugDescription: "No value for key \(key.stringValue)"))
}
return pair.1
}
func decodeNil(forKey key: Key) throws -> Bool { !contains(key) }
func decode(_ type: Bool.Type, forKey key: Key) throws -> Bool { try value(for: key) == "true" }
func decode(_ type: String.Type, forKey key: Key) throws -> String { try value(for: key) }
func decode(_ type: Int.Type, forKey key: Key) throws -> Int {
guard let v = Int(try value(for: key)) else {
throw DecodingError.typeMismatch(Int.self,
.init(codingPath: codingPath + [key], debugDescription: "Not an Int"))
}
return v
}
func decode(_ type: Double.Type, forKey key: Key) throws -> Double {
guard let v = Double(try value(for: key)) else {
throw DecodingError.typeMismatch(Double.self,
.init(codingPath: codingPath + [key], debugDescription: "Not a Double"))
}
return v
}
// ... Int8, Int16, Int32, Int64, UInt, UInt8, UInt16, UInt32, UInt64, Float
// Each one follows the same pattern as Int/Double above.
func decode(_ type: Float.Type, forKey key: Key) throws -> Float {
guard let v = Float(try value(for: key)) else {
throw DecodingError.typeMismatch(Float.self,
.init(codingPath: codingPath + [key], debugDescription: "Not a Float"))
}
return v
}
func decode(_ type: Int8.Type, forKey key: Key) throws -> Int8 { /* same pattern */ }
func decode(_ type: Int16.Type, forKey key: Key) throws -> Int16 { /* same pattern */ }
func decode(_ type: Int32.Type, forKey key: Key) throws -> Int32 { /* same pattern */ }
func decode(_ type: Int64.Type, forKey key: Key) throws -> Int64 { /* same pattern */ }
func decode(_ type: UInt.Type, forKey key: Key) throws -> UInt { /* same pattern */ }
func decode(_ type: UInt8.Type, forKey key: Key) throws -> UInt8 { /* same pattern */ }
func decode(_ type: UInt16.Type, forKey key: Key) throws -> UInt16 { /* same pattern */ }
func decode(_ type: UInt32.Type, forKey key: Key) throws -> UInt32 { /* same pattern */ }
func decode(_ type: UInt64.Type, forKey key: Key) throws -> UInt64 { /* same pattern */ }
func decode<T: Decodable>(_ type: T.Type, forKey key: Key) throws -> T {
if type == String.self { return try decode(String.self, forKey: key) as! T }
throw DecodingError.typeMismatch(type,
.init(codingPath: codingPath + [key], debugDescription: "Nested decoding not supported"))
}
func nestedContainer<NK: CodingKey>(keyedBy type: NK.Type, forKey key: Key) throws -> KeyedDecodingContainer<NK> {
throw DecodingError.typeMismatch(type,
.init(codingPath: codingPath, debugDescription: "Nested containers not supported"))
}
func nestedUnkeyedContainer(forKey key: Key) throws -> UnkeyedDecodingContainer {
throw DecodingError.typeMismatch([Any].self,
.init(codingPath: codingPath, debugDescription: "Nested containers not supported"))
}
func superDecoder() throws -> Decoder { _FormUrlDecoder(pairs: pairs, codingPath: codingPath) }
func superDecoder(forKey key: Key) throws -> Decoder { _FormUrlDecoder(pairs: pairs, codingPath: codingPath + [key]) }
}option 2) in-line parser looking for '&'s and '='s
private static func parseFormEncoded(_ data: Data, warnings: inout [NetworkBodyWarning]) -> Body {
guard let string = String(data: data, encoding: .utf8) ?? String(data: data, encoding: .isoLatin1) else {
warnings.append(.bodyParseError)
return parseText(data, warnings: &warnings)
}
var formData = [String: String]()
for pair in string.split(separator: "&", omittingEmptySubsequences: true) {
let parts = pair.split(separator: "=", maxSplits: 1)
let key = String(parts[0]).removingPercentEncoding?.replacingOccurrences(of: "+", with: " ") ?? String(parts[0])
let value = parts.count > 1
? (String(parts[1]).removingPercentEncoding?.replacingOccurrences(of: "+", with: " ") ?? String(parts[1]))
: ""
if !key.isEmpty { formData[key] = value }
}
return Body(content: formData, warnings: warnings)
}There was a problem hiding this comment.
Also you assume that the body type is either UTF-8 or ISO-Latin-1, but http requests should actually have a content-type header which defines the encoding format.
See #7585 (comment)
There was a problem hiding this comment.
@43jay thanks for research and the proposed options. Reviewing code implementation in a PR review comments is hard to do, so I can't really tell right now which approach is better.
But I just remembered I had a similar issue in the past and created a Form URL Encoder/Decoder we could vendor in from my networking library and add tests:
https://github.com/kula-app/Postie/tree/main/Sources/URLEncodedFormCoding
It should already be fully functional but not fully tested.
There was a problem hiding this comment.
draft while i address this
There was a problem hiding this comment.
@philprime looking at the library you linked, majority of it is for decoding into arbitrary Decodable types.
B/c we'll only ever need to parse [String: String], i opted to just this specific parsing logic from postie instead of the whole shebang.
Delta:
- converts duplicate keys into an array (from postie)
- converts '+' character to ' ' character (new; 1 of the reported issues w/ URLComponents)
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Outdated
Show resolved
Hide resolved
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Outdated
Show resolved
Hide resolved
7db5474 to
a75e679
Compare
#7585 (comment) Parse MIME type and charset from Content-Type using CFStringConvertIANACharSetNameToEncoding instead of hardcoding UTF-8/ISO-Latin-1 fallbacks.
fbd791c to
e586a59
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: JSON fallback to text ignores charset encoding
- The bug was real and fixed by passing the parsed charset encoding into JSON parsing and its text fallback path so malformed JSON now decodes with the declared charset.
Or push these changes by commenting:
@cursor push 318ddc50df
Preview (318ddc50df)
diff --git a/Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift b/Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
--- a/Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
+++ b/Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
@@ -128,7 +128,7 @@
}
if utType.conforms(to: .json) {
if isTruncated { warnings.append(.jsonTruncated) }
- return parseJSON(data, warnings: &warnings)
+ return parseJSON(data, encoding: encoding, warnings: &warnings)
}
if utType.conforms(to: .text) {
if isTruncated { warnings.append(.textTruncated) }
@@ -137,13 +137,13 @@
return nil
}
- private static func parseJSON(_ data: Data, warnings: inout [NetworkBodyWarning]) -> Body {
+ private static func parseJSON(_ data: Data, encoding: String.Encoding, warnings: inout [NetworkBodyWarning]) -> Body {
do {
let json = try JSONSerialization.jsonObject(with: data, options: .mutableContainers)
return Body(content: json, warnings: warnings)
} catch {
warnings.append(.bodyParseError)
- return parseText(data, warnings: &warnings)
+ return parseText(data, encoding: encoding, warnings: &warnings)
}
}This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Outdated
Show resolved
Hide resolved
1) Extracts bodies that are JSON, formurlencoded, or text. Uses UTType to accurately classify content types as JSON or text without maintaining a manual list. Falls back to a string match for application/x-www-form-urlencoded which has no UTType representation. !Relies on having a valid `contentType` 2) Populates NetworkBodyWarning's for "MAYBE_JSON_TRUNCATED" "TEXT_TRUNCATED" "BODY_PARSE_ERROR" ^when encountered, these show custom dashboard UI.
Uses UTType to classify content types: only content positively identified as text is decoded. Everything else gets a descriptive placeholder: Example - "[Body not captured: contentType=image/png (8 bytes)]" Known text types (where UTType conforms to .text) are reliably classified by UTType's type hierarchy. If a content type header is incorrect (e.g. claims text but contains binary), the resulting decode failure is caught by the existing bodyParseError warning.
UTType (UniformTypeIdentifiers) requires macOS 11+, but the SDK targets macOS 10.14. Extract UTType-based MIME detection into a separate method gated with @available(macOS 11, *) so the code compiles on all macOS targets. Session Replay is not available on macOS, so the fallback placeholder is fine.
Casts to lower-case before comparing headers. ObjC setters now accept raw allHeaders and configuredHeaders instead of pre-filtered headers, keeping the filtering logic in Swift.
#7585 (comment) Parse MIME type and charset from Content-Type using CFStringConvertIANACharSetNameToEncoding instead of hardcoding UTF-8/ISO-Latin-1 fallbacks.
a75e679 to
75d4331
Compare
e586a59 to
37167c7
Compare
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Show resolved
Hide resolved
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Show resolved
Hide resolved
5114950 to
bb1abf8
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Sources/Swift/Integrations/SessionReplay/SentryReplayNetworkDetails.swift
Outdated
Show resolved
Hide resolved
philprime
left a comment
There was a problem hiding this comment.
LGTM, I'll approve it as I trust you can apply the final cleanup.
| /// UTType requires macOS 11+; so this will not compile there. | ||
| @available(macOS 11, *) | ||
| private static func parseByMimeType(_ mimeType: String?, data: Data, encoding: String.Encoding, isTruncated: Bool, warnings: inout [NetworkBodyWarning]) -> Body? { | ||
| guard let utType = mimeType.flatMap({ UTType(mimeType: $0) }) else { |
There was a problem hiding this comment.
l: [nitpick] Using flatMap for nul unwrapping in a guard statement is an anti-pattern.
| guard let utType = mimeType.flatMap({ UTType(mimeType: $0) }) else { | |
| guard let mimeType = mimeType, let utType = UTType(mimeType: mimeType) else { |
|
|
||
| func testInit_withFormURLEncoded_duplicateKeys_shouldPromoteToArray() throws { | ||
| // -- Act -- | ||
| let body = try XCTUnwrap(Body( |
There was a problem hiding this comment.
m: Don't use force-unwrapping, we use XCTUnwrap for that, this needs to be fixed in all tests
| if let bodyString = result?["body"] as? String { | ||
| XCTAssertTrue(bodyString.hasPrefix("[Body not captured")) | ||
| } else { | ||
| XCTFail("Expected body to be a string with binary data prefix") | ||
| } |
There was a problem hiding this comment.
m: XCTFail in an if-statement for null unwrapping should be XTCUnwrap.
| if let bodyString = result?["body"] as? String { | |
| XCTAssertTrue(bodyString.hasPrefix("[Body not captured")) | |
| } else { | |
| XCTFail("Expected body to be a string with binary data prefix") | |
| } | |
| let bodyString = try XCTUnwrap(result?["body"] as? String, "Expected body to be a string with binary data prefix") | |
| XCTAssertTrue(bodyString.hasPrefix("[Body not captured")) |
This applies to multiple tests


📜 Description
SentryNetworkBodyExtract request/response bodies that are either JSON, formurlencoded, binary or text.
contentTypeto decide how to interpret the body (NSData).UTTypeto decide whether something is JSON or text.text:
tries
.utf8, then.isoLatin1then gives up withBODY_PARSE_ERRORwarning.json:
tries
JSONSerialization.jsonObject, gives up withBODY_PARSE_ERRORwarningUTType doesn't know:
extracts a placeholder body E.g.
"[Body not captured: contentType=application/octet-stream (10240 bytes)]"SentryReplayNetworkRequestOrResponse💡 Motivation and Context
See first PR in stack.
💚 How did you test it?
Unit tests
make test-ios ONLY_TESTING="SentryReplayNetworkDetailsBodyTests,SentryReplayNetworkDetailsHeaderTests,SentryReplayNetworkDetailsIntegrationTests"Test Suite 'SentryReplayNetworkDetailsBodyTests' started at 2026-03-19 15:25:09.315.
✔ testInit_withBinaryContentType_shouldCreateArtificialString (0.005 seconds)
✔ testInit_withEmptyData_shouldReturnNil (0.000 seconds)
✔ testInit_withFormURLEncoded_duplicateKeys_shouldPromoteToArray (0.001 seconds)
✔ testInit_withFormURLEncoded_emptyKeys_shouldBeSkipped (0.000 seconds)
✔ testInit_withFormURLEncoded_emptyValue_shouldParseAsEmptyString (0.000 seconds)
Resolving Package Graph
✔ testInit_withFormURLEncoded_equalsInValue_shouldPreserve (0.000 seconds)
✔ testInit_withFormURLEncoded_missingEquals_shouldFallbackToText (0.001 seconds)
✔ testInit_withFormURLEncoded_shouldParseAsForm (0.001 seconds)
✔ testInit_withInvalidJSON_shouldFallbackToString (0.001 seconds)
✔ testInit_withJSONArray_shouldParseCorrectly (0.000 seconds)
✔ testInit_withJSONDictionary_shouldParseCorrectly (0.001 seconds)
✔ testInit_withLargeData_shouldTruncate (0.005 seconds)
✔ testInit_withNilContentType_shouldCreatePlaceholder (0.000 seconds)
✔ testInit_withTextData_shouldStoreAsString (0.000 seconds)
✔ testInit_withUnrecognizedContentType_shouldCreatePlaceholder (0.000 seconds)
✔ testParseMimeAndEncoding_shouldHandleEdgeCases (0.001 seconds)
✔ testSerialize_withJSONArray_shouldReturnArray (0.001 seconds)
✔ testSerialize_withJSONDictionary_shouldReturnDictionary (0.001 seconds)
✔ testSerialize_withNoContentType_shouldCreatePlaceholder (0.001 seconds)
✔ testSerialize_withStringBody_shouldReturnDictionary (0.001 seconds)
Executed 20 tests, with 0 failures (0 unexpected) in 0.021 (0.041) seconds
Test Suite 'SentryTests.xctest' passed at 2026-03-19 15:25:09.356.
Executed 20 tests, with 0 failures (0 unexpected) in 0.021 (0.041) seconds
SentryReplayNetworkDetailsHeaderTests (4 tests):
testExtractHeaders_withNilInputs_returnsEmptyDict— nil headers/config returns emptytestExtractHeaders_unconfiguredHeadersAreExcluded— only configured headers are extractedtestExtractHeaders_caseInsensitiveMatching— header matching is case-insensitivetestExtractHeaders_nonStringValues_convertedToStrings— non-string header values are convertedSentryReplayNetworkDetailsIntegrationTests (4 tests):
testInit_withMethod_shouldSetMethod— HTTP method is storedtestSerialize_withFullData_shouldReturnCompleteDictionary— full detail serializes all fieldstestSerialize_withPartialData_shouldOnlyIncludeSetFields— partial data omits unset fieldstestSerialize_withHeaderFiltering_shouldOnlyIncludeConfiguredHeaders— header filtering works end-to-end📝 Checklist
You have to check all boxes before merging:
sendDefaultPIIis enabled. N/ACloses #7710