Conversation
"Reference": https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html Validated against AWS S3.
There was a problem hiding this comment.
Pull request overview
Adds support for additional checksum algorithms (MD5, SHA-512, XXHash64, XXH3-64, XXH3-128) across the client’s S3 request/response handling and functional validation.
Changes:
- Extend
ChecksumTypeto include MD5/SHA512/XXHash variants and provide hashing/size/header-key support. - Propagate new checksum fields through object/part data structures and response parsing paths (Put, multipart, list versions, object attributes).
- Expand functional tests to exercise the new checksum types; update example modules to Go 1.25 and add required hashing dependencies.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| utils.go | Include new checksum headers when building ObjectInfo from response headers. |
| checksum.go | Add new checksum types, header keys, hashers, raw sizes, and string representations. |
| api-datatypes.go | Add new checksum fields to UploadInfo and ObjectInfo. |
| api-s3-datatypes.go | Extend XML datatypes (versions, parts, multipart results) with new checksum fields. |
| api-get-object-attributes.go | Extend GetObjectAttributes checksum XML fields and mapping helpers. |
| api-put-object.go | Carry new per-part checksum fields through multipart stream flows. |
| api-put-object-streaming.go | Carry new per-part checksum fields through streaming multipart flows and PutObject response parsing. |
| api-put-object-multipart.go | Carry new per-part checksum fields and include them in complete-multipart results. |
| api-append-object.go | Include new checksum headers in append response parsing. |
| api-list.go | Populate new checksum fields when listing object versions. |
| functional_tests.go | Add coverage for the new checksum algorithms in put/streaming/multipart checksum tests. |
| go.mod | Add xxhash/v2 and zeebo/xxh3 dependencies. |
| go.sum | Add checksums for new dependencies and transitive modules. |
| examples/s3/go.mod | Align example module to Go 1.25. |
| examples/minio/go.mod | Align example module to Go 1.25. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
f1d5e05 to
c01553c
Compare
|
Obviously running mint fails since it tests against old minio version. |
|
@klauspost we can skip the error in the test if we get "invalid checksum" we can skip the test since this test is downloading OSS minio |
|
Yeah. We have more tests on the server side, so it wouldn't be too much of a loss. I guess we can deal with scenarios like this later. |
|
@harshavardhana Ignoring "InvalidArgument" specifically. Added a const we can flip later if we start testing on "AIStor Free". |
"Reference": https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html
Validated against AWS S3.