Skip to content

Implement provider for azure blob storage#466

Merged
felliott merged 2 commits intoCenterForOpenScience:feature/azure-blob-storagefrom
anqiuy:feature/azure-blob-storage
Aug 28, 2025
Merged

Implement provider for azure blob storage#466
felliott merged 2 commits intoCenterForOpenScience:feature/azure-blob-storagefrom
anqiuy:feature/azure-blob-storage

Conversation

@anqiuy
Copy link
Copy Markdown
Contributor

@anqiuy anqiuy commented Aug 14, 2025

Ticket

NII Feature development Waterbutler
#44

Purpose

Add Azure Blob Storage provider support to WaterButler with OAuth2 authentication and full CRUD operations.

Changes

  • Implemented Azure Blob Storage provider with OAuth2 authentication
  • Added comprehensive metadata handling for files and folders
  • Implemented folder simulation using .osfkeep files
  • Added multi-part upload support and proper error handling
  • Created test cases with Azure API-compliant fixtures

Side effects

None. New provider implementation does not affect existing providers.

QA Notes

Deployment Notes

@anqiuy anqiuy force-pushed the feature/azure-blob-storage branch from 679b0fd to 7e0a94b Compare August 14, 2025 10:26
@anqiuy
Copy link
Copy Markdown
Contributor Author

anqiuy commented Aug 14, 2025

@felliott could you please help review my changes for azure blob storage?

@anqiuy anqiuy force-pushed the feature/azure-blob-storage branch from ca6525b to 3b6fdc0 Compare August 19, 2025 23:30
@felliott
Copy link
Copy Markdown
Member

I've merged develop into feature/azure-blob-storage to help condense the diff here. I will review it today.

@anqiuy
Copy link
Copy Markdown
Contributor Author

anqiuy commented Aug 21, 2025

I've merged develop into feature/azure-blob-storage to help condense the diff here. I will review it today.

Thank you so much for your help. The PR now only includes the changes related to Azure Blob, which makes it much clearer.

@anqiuy
Copy link
Copy Markdown
Contributor Author

anqiuy commented Aug 22, 2025

@felliott Could you help review it today? We'd like to start end-to-end testing for this feature in the staging environment as soon as possible.
Also, the change in CenterForOpenScience/osf.io#11255 is intended to fix an issue where the API service couldn't correctly resolve the gravyvalet addon. After confirming with Brian J. Geiger and @adlius, we decided to apply this fix specifically for the AzureBlobStorage file node. As I understand it, this is meant to be a temporary solution, and a more unified approach to handling the gravyvalet addon will be implemented in the future.

Copy link
Copy Markdown
Member

@felliott felliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Great job on the test suite, you did an excellent job on making them comprehensive and understandable. I have a few small questions and nitpicks, but nothing too substantive. There are a few small flake errors that will prevent CI from running, but once those are fixed, i think it's good to merge. Thank you!

Cheers,
@felliott

@anqiuy anqiuy force-pushed the feature/azure-blob-storage branch from 61304c7 to 562e9d8 Compare August 28, 2025 14:12
@anqiuy anqiuy requested a review from felliott August 28, 2025 14:13
@coveralls
Copy link
Copy Markdown

Coverage Status

coverage: 84.442% (-0.3%) from 84.753%
when pulling 562e9d8 on anqiuy:feature/azure-blob-storage
into aae81c9 on CenterForOpenScience:feature/azure-blob-storage.

@felliott felliott merged commit 1e39d08 into CenterForOpenScience:feature/azure-blob-storage Aug 28, 2025
3 of 4 checks passed
@felliott
Copy link
Copy Markdown
Member

Thanks @anqiuy ! The changes look good, and I have merged the branch.

Cheers,
@felliott

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants