feat(price_validate): add price validation tool and automated pricing CI#231
Draft
dm3ch wants to merge 14 commits intocloudpilot-ai:mainfrom
Draft
feat(price_validate): add price validation tool and automated pricing CI#231dm3ch wants to merge 14 commits intocloudpilot-ai:mainfrom
dm3ch wants to merge 14 commits intocloudpilot-ai:mainfrom
Conversation
Contributor
|
Thanks to your contribution, the maintainers will review it as soon as they can! |
gitautomator
bot
added
enhancement
dm3ch
force-pushed
the
feat/price-validator
branch
from
83f598b to
c5d3c6e
Compare
dm3ch
force-pushed
the
feat/price-validator
branch
from
8b0d4c7 to
f225b34
Compare
…ased implementation
Extract GCP instance pricing into pkg/providers/pricing/instanceprice/ with a
clean interface (New, Close, FetchPrices, IsBlacklisted) shaped for the
upcoming SKU-based rewrite. The Cyclenerd CSV implementation satisfies the
interface now; the rewrite will swap internals without changing the surface.
pricing.go is updated to use instanceprice.Client and adopts cleaner patterns:
- Single sync.RWMutex (was separate muOnDemand/muSpot)
- All JSON parsing / map building outside the lock; only the final pointer
swap is guarded
- LivenessProbe returns an error when no prices are loaded (was a no-op)
- pricesStorage is a defined type, not an alias
initial-prices.json replaces initial-on-demand-prices.json with the
{saved_at, prices:{region:{machine:{on_demand,spot}}}} format that
price_validate and the update-pricing CI workflow produce, so make
update-pricing becomes a plain file copy with no conversion step.
pricing/controller.go names the refresh interval constant and drops the
unnecessary multierr wrapper around a single UpdatePrices call.
Signed-off-by: Dmitry Chepurovskiy <[email protected]>
dm3ch
force-pushed
the
feat/price-validator
branch
14 times, most recently
from
ad9ddbf to
e5a31f0
Compare
dm3ch
commented
Apr 13, 2026
Introduces the price_validate tool that compares computed instance prices against two independent reference sources (Cyclenerd CSV and GCP web pricing). Also adds update-pricing.yaml GitHub Actions workflow that runs weekly and opens a PR when initial-prices.json needs updating. Key design decisions: - Use *time.Time with omitempty for saved_at so computed.json omits the timestamp entirely — only cache files carry it. This prevents spurious weekly PRs when prices have not actually changed. - price_validate exits 0 always; non-zero exit for mismatches is deferred to the upcoming SKU-based pricing rewrite (tracked in cloudpilot-ai#218). - Only roles/compute.viewer is required for the CI service account; the Billing Catalog API is public and needs no IAM grant. Signed-off-by: Dmitry Chepurovskiy <[email protected]>
dm3ch
force-pushed
the
feat/price-validator
branch
from
9a20369 to
89fd8b5
Compare
Collaborator
Author
E2E Test ResultsBranch: feat/price-validator
All executed specs passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
/kind cleanup
What this PR does / why we need it:
This PR refactors current pricing logic and updates initial pricing logic. It's split from #218 and needed to introduce our own instance price calculation (based on GCP SKUs) later, which that PR would bring.
Three things in one PR (they're tightly coupled):
instancepricepackage andpricing.gochanges to use this package - extracts Cyclenerd CSV pricing intopkg/providers/pricing/instanceprice/with a stable interface, ready for the SKU-based rewrite in PR DRAFT: Calculate machine type prices based on CloudBilling SKUs instead of relying on 3rd party data source #218.price_validatetool - This tool would be used for update of the initial-prices file and validating ourinstancepriceresults over data from Cyclenerd and GCP website sources, to help to catch price mismatches..github/workflows/update-pricing.yamlCI job - to automatically create PRs with updated initial prices if they changed according toinstancepricepackage.Which issue(s) this PR fixes:
Related #33
Special notes for your reviewer:
initial-prices.jsonas part of this PR.price_validatenow will report a high number of mismatches. It's expected and fine. That happens because during validation we treat pricing info from GCP website as more trustworthy compared to Cyclenerd pricing info. These mismatches would be fixed by new instanceprice implementation that exists in DRAFT: Calculate machine type prices based on CloudBilling SKUs instead of relying on 3rd party data source #218.hack/tools/price_validate/gcpweb.gois a little bit messy, as we parse data from JS blocks of HTML pages. I don't know how to make it easier to read. But I assume that's fine, cause it's isolated code in a supplementary tool..github/workflows/update-pricing.yamlfile.Does this PR introduce a user-facing change?