[Core] raw githubusercontent urls are updated to refer azcli blob to restrict external system access#9826
Conversation
…t external system access
️✔️Azure CLI Extensions Breaking Change Test
|
|
Hi @msarfraz, |
|
Thank you for your contribution! We will review the pull request and get back to you soon. |
|
The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR. Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions). pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>
|
CodeGen Tools Feedback CollectionThank you for using our CodeGen tool. We value your feedback, and we would like to know how we can improve our product. Please take a few minutes to fill our codegen survey |
|
There was a problem hiding this comment.
Pull request overview
This PR aims to remove reliance on raw.githubusercontent.com for VM image alias lookups by switching references to azcliprod.blob.core.windows.net, and adds a CI guard to prevent introducing new raw GitHub URLs in future diffs.
Changes:
- Updated multiple test recordings to replace
raw.githubusercontent.com/.../aliases.jsonwithhttps://azcliprod.blob.core.windows.net/cli/vm/aliases*.json. - Added a CI script (
validate_external_source_urls.py) plus an exclusions configuration to flag new uses ofraw.githubusercontent.comin added diff lines. - Added a new Azure Pipelines job to run the external-URL validation on PR builds.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/vme/azext_vme/tests/latest/recordings/test_vme_upgrade_live.yaml | Updates recorded vmImageAliasDoc URL to the internal blob-hosted alias file. |
| src/vme/azext_vme/tests/latest/recordings/test_vme_live.yaml | Updates recorded vmImageAliasDoc URL to the internal blob-hosted alias file (multiple occurrences). |
| src/storage-mover/azext_storage_mover/tests/latest/recordings/test_storage_mover_job_definition_scenarios.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/storage-mover/azext_storage_mover/tests/latest/recordings/test_storage_mover_endpoint_scenarios.yaml | Rewrites recorded alias JSON fetch URL to blob storage (multiple occurrences). |
| src/site-recovery/azext_site_recovery/tests/latest/recordings/test_siterecovery_scenarios.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/site-recovery/azext_site_recovery/tests/latest/recordings/test_siterecovery_A2A_selfcreated_scenarios.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/serial-console/azext_serialconsole/tests/latest/recordings/test_check_resource_VMSS.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/serial-console/azext_serialconsole/tests/latest/recordings/test_check_resource_VM.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/scheduled-query/azext_scheduled_query/tests/latest/recordings/test_scheduled_query_update_action_group.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/scheduled-query/azext_scheduled_query/tests/latest/recordings/test_scheduled_query_condition_operator.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/monitor-control-service/azext_amcs/tests/latest/recordings/test_monitor_control_service_commands.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/monitor-control-service/azext_amcs/tests/latest/recordings/test_amcs_data_collection_endpoint_association.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/logz/azext_logz/tests/latest/recordings/test_sub_account_vm.yaml | Updates recorded alias URL from master raw GitHub to blob-hosted aliases_master.json. |
| src/logz/azext_logz/tests/latest/recordings/test_main_account_vm.yaml | Updates recorded alias URL from master raw GitHub to blob-hosted aliases_master.json. |
| src/image-gallery/azext_image_gallery/tests/latest/recordings/test_create_vm_with_community_gallery_image.yaml | Updates recorded alias URL from master raw GitHub to blob-hosted aliases_master.json. |
| src/image-gallery/azext_image_gallery/tests/latest/recordings/test_community_gallery_operations.yaml | Updates recorded alias URL from master raw GitHub to blob-hosted aliases_master.json. |
| src/hardware-security-modules/azext_hardware_security_modules/tests/latest/recordings/test_hardwaresecuritymodules.yaml | Updates recorded alias URL from master raw GitHub to blob-hosted aliases_master.json. |
| src/alias/setup.py | Adds new package metadata field (currently introduces a raw GitHub URL). |
| src/aem/azext_aem/tests/latest/recordings/test_WithUserAssignedIdentity.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/aem/azext_aem/tests/latest/recordings/test_OldExtensionReinstall.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/aem/azext_aem/tests/latest/recordings/test_NewExtensionUltraDisk.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/aem/azext_aem/tests/latest/recordings/test_NewExtensionDiskAdd.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| src/aem/azext_aem/tests/latest/recordings/test_ExtensionUpgrade.yaml | Rewrites recorded alias JSON fetch URL to blob storage. |
| scripts/ci/validate_external_source_urls.py | New CI validator to detect forbidden raw GitHub URLs in added diff lines. |
| scripts/ci/external_url_exclusions.json | New configuration defining path-based exclusions for the validator. |
| azure-pipelines.yml | Adds a new PR job to run the external URL validation. |
|
Please fix CI issues |
This reverts commit 4f284fb.
|
@naga-nandyala @wangzelin007 please help with this PR review |
Description
This PR removes the dependency on GitHub (raw.githubusercontent.com) VM image aliases, replacing it with Azure Blob Storage (azcliprod.blob.core.windows.net). This change enables Azure CLI to work properly in network isolated environments where GitHub access is blocked.
In addition, new validation added in CI pipeline to flag if any raw.githubusercontent.com URL is used in any upcoming PR code change.
Background
In enterprise environments with strict network isolation policies, access to raw.githubusercontent.com is not allowed.
Changes
VM Image Alias Migration
Before:
https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/arm-compute/quickstart-templates/aliases.json
After:
https://azcliprod.blob.core.windows.net/cli/vm/aliases_master.json
Before:
https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/arm-compute/quickstart-templates/aliases.json
After:
https://azcliprod.blob.core.windows.net/cli/vm/aliases.json
This checklist is used to make sure that common guidelines for a pull request are followed.
Related command
General Guidelines
azdev style <YOUR_EXT>locally? (pip install azdevrequired)python scripts/ci/test_index.py -qlocally? (pip install wheel==0.30.0required)For new extensions:
About Extension Publish
There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update
src/index.jsonautomatically.You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify
src/index.json.