Skip to content

Commit e22a92b

Browse files
authored
Chore: upgrade dependencies, add upgrade sklill (#56)
1 parent 3d55754 commit e22a92b

8 files changed

Lines changed: 387 additions & 150 deletions

File tree

.bumpversion.cfg

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[bumpversion]
2-
current_version = 0.4.0
2+
current_version = 0.4.1
33
commit = True
44
tag = False
55

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
name: python-upgrade
3+
description: Audit and upgrade Python dependencies to fix vulnerabilities and apply patch-level updates, with test verification.
4+
allowed-tools: Bash, Read, Edit, Write, Agent, WebFetch, WebSearch
5+
user-invocable: true
6+
---
7+
8+
Our Python dependencies have vulnerabilities that we want to fix by upgrading.
9+
10+
## Setup
11+
12+
Switch to git branch "chore/upgrade", creating it if it doesn't exist.
13+
Start with `poetry sync --with dev` to be up to date with the lock file.
14+
15+
## Auditing
16+
17+
- Run `poetry run pip-audit` to list vulnerabilities.
18+
- Run `poetry show --outdated` to see packages that can be upgraded in general.
19+
20+
## Upgrading
21+
22+
- Use `poetry update <package-name>` to upgrade a dependency within the version range in pyproject.toml.
23+
- Use `poetry add <package>` to upgrade a package to something outside the version range in pyproject.toml.
24+
- Avoid code changes unless the fix is trivial (e.g., an import path change). If code changes are needed, note them in the summary.
25+
- Transitive dependencies can be upgraded if they have a security vulnerability. Note any upgraded transitive dependencies in the summary.
26+
- Ask before upgrading pinned versions.
27+
28+
## Severity lookup
29+
30+
Severity information for each vulnerability can be found like so https://github.com/advisories?query=CVE-2025-14009. Look up each advisory to determine its severity (critical, high, medium, low).
31+
32+
## Upgrade strategy
33+
34+
1. Run `pip-audit` and collect all vulnerabilities.
35+
2. Look up the severity of each vulnerability.
36+
3. Display a plan as a table showing each vulnerability with: package name, current version, fixed version, advisory ID, severity, depndency group (for exmaple "dev"), and the semantic version increase required.
37+
4. Sort the plan by: severity (critical first), then by smallest semantic version increase first.
38+
5. Perform all upgrades in the plan.
39+
6. Run `poetry run pytest` to see if the upgrades were successful.
40+
7. If all tests passed, perform all other outstanding upgrades if they are at patch level and within pyproject.toml ranges.
41+
8. Run `poetry run pytest` to see if all upgrades were successful.
42+
9. If any upgrade fails or if any test fails, roll back all changes with `git checkout -- poetry.lock pyproject.toml && poetry sync --with dev` and use the "Careful Upgrade Strategy".
43+
44+
## Careful Upgrade Strategy
45+
46+
1. Use this strategy if the general upgrade strategy fails.
47+
2. Use the plan created by the general upgrade strategy.
48+
3. Upgrade dependencies **one by one**, running `poetry run pytest` after each upgrade to ensure everything still works.
49+
4. If a test fails after an upgrade, revert that upgrade and move on to the next one. Note which upgrades were skipped and why.
50+
5. It might not be possible to upgrade all dependencies. Use your best effort.
51+
52+
Think hard about dependency interactions and potential breaking changes.
53+
54+
## Integration test
55+
56+
After all achievable upgrades are complete, run a final integration test by starting the local server and sending a test GraphQL query:
57+
58+
1. Run `yarn install` if node_modules are not present.
59+
2. Temporarily change `pythonBin: python3` to `pythonBin: python` in `serverless.yml` (needed for Windows compatibility).
60+
3. Start the server in the background, wait for it to be ready, then send a query:
61+
62+
```bash
63+
ENABLE_METRICS=0 poetry run yarn sls wsgi serve > /tmp/server.log 2>&1 &
64+
SERVER_PID=$!
65+
```
66+
67+
Then in a separate command, poll the log file until the server is ready before sending the query. The server is ready when the log contains "Serving on". Retry up to 10 times with 3-second sleeps:
68+
69+
```bash
70+
for i in $(seq 1 10); do
71+
if grep -q "Serving on" /tmp/server.log 2>/dev/null; then
72+
echo "Server is ready"
73+
break
74+
fi
75+
echo "Waiting for server... (attempt $i)"
76+
sleep 3
77+
done
78+
79+
cat /tmp/server.log
80+
81+
curl -s -X POST http://localhost:5000/graphql \
82+
-H "Content-Type: application/json" \
83+
-d '{"query": "{ version about get_models { version } }"}' 2>&1
84+
```
85+
86+
If the server log does not contain "Serving on" after all retries, print the log contents and report the failure instead of sending the query.
87+
88+
Then clean up:
89+
90+
```bash
91+
kill $SERVER_PID 2>/dev/null
92+
```
93+
94+
4. Verify the response contains valid `version`, `about`, and `get_models` data.
95+
5. Revert the `pythonBin` change in `serverless.yml` back to `python3`.
96+
97+
## Bump version
98+
99+
When done, create a commit and bump the version with
100+
101+
```bash
102+
poetry run bump2version patch
103+
```

.gitignore

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,5 +114,7 @@ ENV/
114114

115115
# mkdocs build dir
116116
site/
117+
117118
node_modules
118-
node_modules
119+
.yarn/install-state.gz
120+
.claude/settings.local.json

CLAUDE.md

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# CLAUDE.md
2+
3+
## Project Overview
4+
5+
A GraphQL API exposing New Zealand Seismic Hazard Model (NSHM) data via AWS Lambda. Built with Python/Flask/Graphene, deployed using Serverless Framework v4.
6+
7+
## Common Commands
8+
9+
```bash
10+
# Install dependencies
11+
yarn install
12+
poetry install
13+
14+
# Run tests
15+
poetry run pytest
16+
poetry run pytest tests/test_schema_models.py # single test file
17+
poetry run pytest tests/test_schema_models.py::test_fn # single test function
18+
poetry run pytest --cov # with coverage
19+
20+
# Lint & format
21+
poetry run black nshm_model_graphql_api tests
22+
poetry run isort nshm_model_graphql_api tests
23+
poetry run flake8 nshm_model_graphql_api tests
24+
poetry run mypy nshm_model_graphql_api tests
25+
26+
# Full CI suite via tox
27+
tox # all environments: audit, py312, format, lint, build
28+
tox -e py312 # tests only
29+
tox -e lint # lint only
30+
31+
# Local dev server (port 5000)
32+
ENABLE_METRICS=0 poetry run yarn sls wsgi serve
33+
34+
# Deploy
35+
AWS_PROFILE=<profile> poetry run yarn sls deploy --region ap-southeast-2 --stage dev
36+
```
37+
38+
## Architecture
39+
40+
**Query-only GraphQL API** — no mutations. The API wraps the `nzshm-model` Python library, which contains the actual NSHM data.
41+
42+
### Schema Structure (`nshm_model_graphql_api/schema/`)
43+
44+
- `schema_root.py` — Root query type with top-level resolvers (`get_models`, `get_model`, `current_model_version`, `about`, `version`, `node`)
45+
- `nshm_model_schema.py``NshmModel` type with nested `source_logic_tree` and `gmm_logic_tree`
46+
- `nshm_model_sources_schema.py` — Source logic tree types: `SourceLogicTree``SourceBranchSet``SourceBranch``InversionSource`/`DistributedSource`
47+
- `nshm_model_gmms_schema.py` — Ground motion model logic tree types: `GmmLogicTree``GmmBranchSet``GmmBranch`
48+
49+
All types implement Relay's `Node` interface for global ID-based lookups. Resolvers use `@functools.lru_cache` for expensive data fetches.
50+
51+
### Flask App
52+
53+
`nshm_model_graphql_api/nshm_model_graphql_api.py` — Flask app factory. Serves GraphQL at `/graphql` (POST for queries, GET for GraphiQL interface).
54+
55+
### Deployment
56+
57+
Serverless Framework v4 deploys to AWS Lambda (Python 3.12, 2048MB, 10s timeout) in `ap-southeast-2`. API Gateway provides HTTP endpoints with API key auth on POST.
58+
59+
- `serverless.yml` — Lambda functions, API Gateway routes, WSGI config
60+
- `package.json` — Serverless CLI and plugins (serverless-wsgi, serverless-plugin-warmup)
61+
62+
## Tech Stack
63+
64+
- **Runtime**: Python 3.12, Node 22 (for Serverless CLI)
65+
- **Package managers**: Poetry (Python), Yarn v4 (Node)
66+
- **Testing**: pytest with Graphene test Client
67+
- **CI/CD**: GitHub Actions (`.github/workflows/`) — tests on PR, deploy on merge to main
68+
- **Version management**: bump2version syncs version across `pyproject.toml`, `package.json`, `__init__.py`
69+
70+
## Configuration
71+
72+
- `pyproject.toml` — Poetry dependencies and project metadata
73+
- `setup.cfg` — pytest, coverage, flake8, mypy, tox, isort settings
74+
- `.bumpversion.cfg` — version bump targets

nshm_model_graphql_api/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22

33
__author__ = """GNS Science"""
44
__email__ = "nshm@gns.cri.nz"
5-
__version__ = "0.4.0"
5+
__version__ = "0.4.1"

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "nshm-model-graphql-api",
3-
"version": "0.4.0",
3+
"version": "0.4.1",
44
"description": "A graphql API for NSHM seismic models",
55
"scripts": {
66
"test": "echo \"Error: no test specified\" && exit 1",

0 commit comments

Comments
 (0)