v0.10.0

Announcements

This release add initial support for CMIP7 datasets.
This is still a work in progress so the list of the attributes that are tracked may change as we integrate the providers.

Changelog

Features

• Added database support for CMIP7 datasets based on the CMIP7 Global Attributes v1.0 specification. (#503)
• Added CMIP7 data requirements support, enabling providers to fetch CMIP6 data from ESGF and translate it to CMIP7 format using the CMIP7 CV converter. (#510)
• Added diagnostic summary introspection and auto-generated documentation for all providers. The ref providers show command now defaults to detailed list format and supports --columns for filtering table output. (#518)

Improvements

• Separated model from observation runs for regional historical diagnostics. (#460)
• Made listing the changed files from a regression test faster. (#514)
• Prepared the monorepo for splitting diagnostic provider packages into independent repositories. Extracted shared test fixtures into a climate-ref[test] pytest plugin, decoupled climate-ref-core from application-level types, and added API surface documentation, versioning strategy, provider compatibility CI, and a copier template for bootstrapping new provider repositories. (#520)

Bug Fixes

• Fixed CMEC bundle dimension validation to use a subset check instead of exact equality, allowing diagnostics with multiple data requirements to have varying output dimensions. (#523)

Trivial/Internal Changes

• #516

Changes

• b31c8a5 Bump version: 0.9.1 → 0.10.0
• 2ff4f58 chore: add changelog entry for PR #523
• 9b809ee fix: use subset check for CMEC bundle dimension validation
• ce56f34 fix(helm): use distinct chart version tags per build context
• 9a04820 chore: remove unnecessary conftest.py from climate-ref-core tests
• 38454c9 fix: address PR review comments
• 0ea9510 fix: improve diff coverage and restrict CI permissions
• a461179 fix(ci): use coverage run for proper entry-point plugin tracking
• 603dae7 fix(ci): skip conftest_plugin in imports-without-extras check
• d22d9d8 chore: add changelog entry for #520
• 9feddb4 ci: add provider compatibility workflow and copier template
• 491912a docs: add API surface, versioning strategy, and contributor guide
• 2121d1c refactor: extract shared test fixtures into climate-ref[test] pytest plugin
• 0953583 chore: add main_log_debug.txt to gitignore and remove existing files
• 57ee249 Apply suggestion from @lewisjared
• f6b876b docs: add changelog entry for PR #518
• 52fdcf4 feat(cli): make list format default for providers show and add --columns option
• 8145dee feat: improve diagnostic docs with admonitions, tabs, and overview by source type
• 7f087aa feat: add diagnostic summary introspection and documentation generation
• 8a758f6 test: improve coverage for CMIP7, solver, and test_cases CLI
• 887242e fix: resolve merge conflict in solver OR-logic after InvalidDiagnosticException removal
• ebd8a0f fix(tests): set up context manager mocks for xr.open_dataset in CMIP7 tests
• 04b7e3f Add changelog
• e81fdef Add codecov.yml with relaxed coverage thresholds
• 7aa967c feat(cmip7): add license_id and external_variables to CMIP7 model
• ce39225 Also update output collection of regional historical trend diagnostic
• f42fac2 Add changelog
• f942b93 Faster listing of regression data
• 9e334b9 Add another variable to test case
• a8fd54b Do not try to push to container registry from forks
• 2bb22bc Update regression test output
• fda40b4 Add changelog
• 3a2083b Split trends recipe and fix the other two regional recipes
• d787258 Update recipes
• c83a5c5 Update regional historical annual cycle and timeseries
• 97242bf fix: address PR review comments
• 32a9b7f test: increase test coverage for CMIP7 data requirements
• 5d09eba docs: add changelog for PR #510
• bcc1754 feat: add CMIP7 data requirements support for Example provider
• 8377a0a style: reorganize import statements for clarity
• 8e47b3e refactor: address PR review comments for CMIP7 support
• 624f40e Add tests for CMIP7 adapter with CMIP6-converted files
• d70e0a2 Add changelog for PR #503
• 0614379 Add CMIP7 database support

v0.9.1

Changelog

Features

• Added ingest_data() lifecycle hook to providers, enabling automatic dataset ingestion during ref providers setup. PMP climatology data is now ingested automatically, eliminating the need for a separate manual ingestion step. (#508)

Improvements

• Implemented coupled versioning for Helm chart: chart version, appVersion, and default image tag now stay in sync with the application version and are updated automatically by bump-my-version. (#507)
• Improved CLI performance by skipping database backup for read-only commands like config list and datasets list. (#511)
• Improved CLI startup time by deferring heavy imports until needed. (#512)

Bug Fixes

• Fixed Helm chart CI to use correct image tag override path (defaults.image.tag instead of invalid climate-ref.image.tag). (#507)

Improved Documentation

• Updated getting started documentation with clearer configuration and dataset download instructions. (#508)

Changes

• b06b4cc Bump version: 0.9.0 → 0.9.1
• c579e31 docs: fix admonition syntax and typo in getting-started guides
• 0d71679 fix(tests): update patch paths for lazy imports
• 6311473 chore: add changelog and fix lint after merge
• 4d500fd test: add unit tests for providers, datasets, config, and models
• ec2352f ci: only run helm jobs when helm/ changes or on main
• ab47504 perf: move SourceDatasetType to lightweight module and add more lazy imports
• 5100872 perf(cli): optimize startup time with lazy imports
• 5b546b0 chore: add changelog for PR #511
• bb10a39 feat(cli): skip database backup for read-only commands
• 758e672 fix: Use the right path
• 8df2043 docs: add changelog entries for PR #508
• c371c79 feat(providers): add ingest_data hook for provider-level dataset ingestion
• 4335793 docs: update configuration and dataset download instructions; remove empty tutorials section
• ba9be9d chore: add skip-validate
• 0086ade docs: add changelog entries for Helm versioning fixes
• a368352 chore: update Helm chart versioning and defaults to 0.9.0
• f2ed7dd chore: bump to latest image
• 7189d4a test(packaging): use the setup command

v0.9.0

Announcements

This release brings some upgrades to the testing framework we use and a streamlined setup process.
This should make it easier to add new diagnostics.

We have temporarily pinned pandas<3 until we can verify that it everything works as expected.

Changelog

Features

• Added test data management infrastructure for diagnostic development:

  • New ref test-cases CLI commands (fetch, list, run) for managing and running diagnostic test cases.
  • ESGF data fetching utilities with support for CMIP6 and obs4MIPs datasets.
  • TestDataSpecification and TestCase classes for defining reproducible test scenarios.

  (#475)

• Add functionality to translate a CMIP6 dataset to the new CMIP7 conventions (#484)

• Added test automation infrastructure for diagnostic testing using test-cases (#485)

• Added CMIP6 to CMIP7 format conversion command-line script to translate CMIP6 datasets into CMIP7-compatible format. (#489)

• Added RegistryRequest class for fetching datasets from pooch registries (pmp-climatology, obs4ref) instead of ESGF. (#490)

• Added Helm chart for Kubernetes deployment with automated CI/CD pipeline for building and publishing the chart to GitHub Container Registry, including deployment templates for provider workloads (ESMValTool, PMP, ILAMB), Flower monitoring UI, Dragonfly Redis dependency, and comprehensive integration testing in minikube. (#492)

• Add CI workflow to verify solve works without network access (ci-offline-solve.yaml).
  This test uses Docker with --network none to block all network access including
  subprocesses. Runs every other day and can be triggered manually. (#497)

• Added provider lifecycle hooks for offline execution setup. Providers can now implement setup_environment(), fetch_data(), and post_setup() methods to prepare for execution on HPC compute nodes without internet access. A new ref providers setup CLI command runs all provider setup hooks, fetching required reference data to the local cache before offline solving. (#498)

Improvements

• Clean up the open database connections in the test suite (#482)
• Improved catalog handling with hash-based change detection and multi-file dataset support. Enhanced CLI test-cases commands with new flags: --only-missing, --force, --dry-run, --if-changed, and --clean. (#490)
• Updated ESMValTool to v2.13.0 (#500)

Trivial/Internal Changes

• #480, #506

Changes

• 2eb538f Bump version: 0.8.1 → 0.9.0
• c435cb4 docs: Changelog
• f8caea7 chore: pin pandas to <3
• 6d3d884 chore(deps-dev): bump the python-dependencies group across 1 directory with 9 updates
• 6e98652 chore: add more test coverage
• eb655a5 Add changelog
• db4ca16 Regenerate example output
• d7ed855 Update ESMValTool to v2.13.0
• b8f7ce6 refactor: address PR review feedback for provider lifecycle
• 2368a2d Update docs/how-to-guides/hpc_executor.md
• 5dcead9 refactor: address PR review feedback for provider lifecycle hooks
• 690f759 docs: update provider setup documentation
• 439a85d revert: remove blocking
• a6c47ae docs: add changelog for provider lifecycle hooks
• 0892d14 fix(config): handle network errors when fetching ignore datasets file
• 665ccd6 feat(providers): add lifecycle hooks for offline execution setup
• c4323d4 revert: makefile
• e477331 test: add Docker-based offline solve test workflow
• 524ea64 chore: don't download all data
• 089de09 fix: add conda environment setup before offline tests
• 2c1a436 fix: fetch test data before running offline solve tests
• 28f1070 test: add CI job to verify solve works without network access
• b0dc7e2 chore: ignore all .ref
• 850c823 chore(deps): bump the github-actions group with 4 updates
• 0d7e81b docs: add documentation for the helm chart
• 62e7768 fix: Set HOME dir to a writable location
• c9aedb0 revert: undo livliness checks for now
• 34b2e16 feat: add liviness probe
• 950b17f chore(deps-dev): bump the python-dependencies group with 3 updates
• 2002e40 chore: Re-enable CI/CD workflows
• 51b163e feat: Build and publish Helm chart in CI
• fd730d3 chore: Add pragma: no cover for display-only CLI code
• 7322e25 refactor: Extract utilities from test_cases.py and add tests
• f878721 docs: Add changelog entries for PR #490
• 149e302 feat: Add RegistryRequest for pooch registries and improve catalog handling
• 724be90 docs: Changelog
• b3d0135 feat: added script to translate CMIP6 to CMIP7
• 22d1eab fix: Updates to understanding from Zeb
• 06a119e chore(deps-dev): bump the python-dependencies group with 4 updates
• d407a89 chore: remove the unneeded files
• 018b53c test: Fix udpates to paths
• a8c5f44 chore: Updates after rereading the global attributes file
• 69b8ec1 chore: use parameterised tests instead
• cb246c0 test: improve coverage
• 0beb3fa Update packages/climate-ref-core/src/climate_ref_core/cmip6_to_cmip7.py
• bb8fefc doc: Changelog
• b2bae88 chore: run test-cases separately
• a4ee861 chore: ignore pytest warning
• bd408dc chore: add example regression output
• 58fd3c9 feat: Add a helper class for tracking test case paths
• 3ea8367 test: offload current testing to the validate cmec bundles routine
• 683fdee chore: run the example diagnostic
• a75e074 chore: add example catalog
• 7ac134a chore: move test case tests to their respective packages
• ce706b4 chore: move catalog to a provider-specific path
• a5e2659 chore: remove doctest as the datafile isn't available
• 83147da docs: Changelog
• daf3c8a feat: Add cmip6 to cmip7 translation
• 7227336 fix: Allow fetching multiple variables
• 5a8b72e fix: typos in docs
• 2e4a7d3 fix: remove unused function
• 236c0b3 chore: improve coverage
• 281c2aa chore: clean up tests
• a27870f feat: Generate a data catalog on fetch
• 6be929d chore: Move ESGF_DATA_DIR definition to testing
• 1f0e89b Update packages/climate-ref/tests/unit/datasets/test_cmip6.py
• 779f974 chore: Fix more warnings
• 51c9b49 chore: allow to fetch and run
• 39d0ef7 chore: Clean up db in the cli
• d9dac70 chore: use pytest 9's strict mode
• e84660c test: Fix old testing value
• 20aeda6 docs: Changelog
• edb2965 chore: Fix sqlite resource errors
• bb2f52a refactor: rename commands
• 7777f24 test: Add more coverage
• 6b850b1 test: test testing tests
• 3b39fab docs: Add documentationf for the new diagnostics
• 77202fc test: improve coverage
• f365d06 doc: Changelog
• 27dde70 revert: Go back to separating stderr and stdout
• 57ea206 chore: pin xarray for ilamb3 due to issue
• 3dc9ca1 Potential fix for pull request finding 'Explicit returns mixed with implicit (fall through) returns'
• 6f70658 fix: handle a regression in xarray
• 84ec4fe fix: Clean up some type hints from updating mypy
• 5326654 docs: fix typo
• e3b2541 feat: Add a harness to run a set of specs
• aafd16d chore(deps-dev): bump the python-dependencies group with 31 updates
• 3b6d9e1 fix: use uv not pip
• 4084648 docs: Changelog
• b7384fb ci: Ignore dependabot PRs
• a1e0121 chore(deps): bump the github-actions group with 5 updates
• bccb710 chore: add ipython dep to fix CI
• 35f8a05 test: Add coverage for the new modules
• d189868 feat: add test spec
• 1a3894c feat: Add esgf fetching utlities from sample repo
• 50ca6bc chore: add required packages

v0.8.1

Changelog

Bug Fixes

• Add a pin for fastprogress (dependency of intake-esm) to work around bug in newer versions (#476)

Trivial/Internal Changes

• #477

Changes

• 7a6c236 Bump version: 0.8.0 → 0.8.1
• 50933b8 docs: Clean up missing changelogs
• 5e51f8f ci: Update uv in CI
• 3f70a7c chore: Add dependabot
• 7caeda5 chore: change to pin for fastprogress
• ef060b7 fix: Bump a yanked version of deprecated
• 9ea1da7 fix: Add IPython to depedencies temporarily

v0.8.0

Changelog

Features

• Enable spatial 3-d variables that has levels for PMP annual cycle. (#411)

• Added ignore datasets constraint and configuration file.

  If no path is configured for the ignore_datasets_file in the configuration file,
  the default ignore datasets file is downloaded from the Climate-REF GitHub repository
  if it does not exist or is older than 6 hours. (#447)

Improvements

• Validated the slurm configurations of the HPCExecutor using pydantic (#375)
• Added reference values to ESMValTool series output. (#452)

Bug Fixes

• General fixes found when rerunning, including handling an edge case where no log output is written,
  ignoring empty input directories and increased logging of the number of executions. (#444)
• Worked around pydata/xarray#2742
  by always replacing the default fillvalue for the data type with NaN in arrays
  read with Xarray. (#454)
• Excluded piControl from PMP annual cycle and variability metrics owing to non-overlapping periods with observations. (#463)

Improved Documentation

• Add a Jupyter notebook showing how to use the CMIP7 Assessment Fast Track website OpenAPI. (#466)

Changes

• 31aa63f Bump version: 0.7.0 → 0.8.0
• f6fe0e8 Fix formatting
• 8b11814 Apply suggestions from code review
• 1229263 Another attempt at having colored output in notebooks
• 478762c Use color in cells
• 364e6d4 More updates
• d5a49f1 rerun ci
• 7aca1de Rename subtitle
• 6d53277 Fix dataset selection notebook
• e49d967 Update dependencies
• 0b66a39 Add changelog
• 0ac7809 Rename
• 0f170d5 Add more examples
• f0a240d Run notebooks
• a408315 Add metric values to the notebook
• 6e8fda5 Add API demo notebook
• 4f24e01 add change log
• 5479e51 removed picontrol
• c3a6874 Add ignore datasets configuration file
• c1cb343 Update reference output
• 735a246 Convert default fillvalues to NaNs
• 218d374 chore: Changelog
• 8b09184 chore: Rename changelog
• 9bbf327 Update packages/climate-ref-pmp/src/climate_ref_pmp/diagnostics/annual_cycle.py
• e589b46 Update reference output
• d58312f Add reference values to series output
• dc2f6ed minor fixes
• 75067cd add docs for the configrations of HPCExecutor validation
• ed442d1 minor cleanup on hpc.py
• 738a68c add NPGO regression test output files
• 5f15381 fix broken test for annual cycle
• 03fd7d3 Lee1043 pmp mov bugfix (#450)
• a5e0623 chore: Catch exception in local
• 515909e add test results
• 6fa4ce8 chore: Minor tweaks to config
• 137b886 test: Fix test
• 0329781 fix: Skip missing directories and continue
• be476cb chore: Add logs for dry-mode
• 7d5190f chore: Add log message for the number of solves
• eb3d33c clean up
• d6d10b7 clean up
• 9d06a32 clean up
• aa81f80 clean up
• 6edda7b add change log
• 6afc821 clean up
• 30688bf update
• 47ba229 update
• aca5dfd in progress
• 3f2ff0f clean up
• 9c3f374 in progress -- push for archiving the progress
• a5c9281 in progress
• a02adda update
• 41a25a5 initial commit
• dfff9d0 add changelog
• ce9e42f validate hpc slurm config using pydantic

v0.7.0

Changelog

Breaking Changes

• Use the directory structure template from the obs4MIPs specification to define
  the instance_id. (#383)

• Used logical or instead of logical and to combine multiple facet filters.

  This means that the selected input datasets will include items that match any of
  the specified FacetFilters, rather than only those that match all of them.
  This change was made to improve the usability of the option to use multiple facet
  filters, allowing users to be more specific with their filtering criteria. (#414)

• Changed all constraints so they return a filtered dataframe instead of a boolean.

  This allows using the constraints in multi-model diagnostics. (#416)

Features

• Add a timerange constraint to ensure the required data is available. (#399)
• Add a human readable representation for climate_ref_core.datasets.ExecutionDatasetCollection. (#401)
• Allow execution outputs to have dimensions.
  These dimensions allow the outputs to be filtered in the same manner as the metric values (#434)
• Adds filter options to the ref executions list-groups command.
  This allows users to filter by facet, provider and diagnostic,
  as well as whether the latest executions was successful
  or if the exection group was has been marked as dirty (#438)
• Added a ref executions delete-groups command for removing unneeded results from the database.
  This will delete the execution group and any executions and outputs associated with the group. (#441)

Improvements

• Updated the sample data to v0.7.3. (#402)
• PMP version updated to v3.9.2 (#404)
• Specified the "table_id" for all ESMValTool diagnostics to improve data selection. (#407)
• Added a groupby operation to the RequireFacets constraint to reduce failed runs because of missing data. (#408)
• Adds extra ESMValTool dimensions to the output series (#410)
• The dataset ingestion process now supports updating datasets if new files become available (#412)
• Added series output for ESMValTool diagnostics. (#413)
• Gracefully handles empty directories when ingesting.
  The default behaviour is now to skip any invalid datasets as that is a sane default when ingesting many datasets. (#419)
• Include obs4MIPs and thetao in the ESGF download script and turn into a CLI tool.
  This script was renamed to fetch-esgf.py to fetch-esgf.py as it now includes more than just CMIP6 data. (#420)
• Improved support for concatenating historical and future experiments in the ESMValTool recipe (#431)
• Log the ESMValTool recipe and configuration for easier debugging (#432)
• Adds output dimensions to ILAMB outputs.
  This adds additional dimensions and attributes to the series values (#435)
• Replaced instance_id by individual facets in group_by of ESMValTool diagnostics. (#439)

Bug Fixes

• Resolves missing index and index names for series (#410)
• Added missing column to the PMP dataset model (#418)
• Removes the ERA-5 ta data Obs4REF in preference for Obs4MIPs (#421)
• Avoid selecting the areacella variable when filling the recipe for cloud scatterplots. (#427)
• Fixes to the ocean heat content IOMB diagnostic (#433)

Trivial/Internal Changes

• #440

Changes

• e146ffb Bump version: 0.6.6 → 0.7.0
• 1864139 chore: Increase coverage
• 7403638 chore: Test other models
• f25a960 feat: Support removing outputs
• 2364218 docs: Changelog
• f290d13 doc: Changelog
• be1f2d2 feat: Add delete command
• 9819515 chore: Improve coverage
• f8ef3fa Remove outdated example output
• 5cbea59 chore: Fix ruff issues
• 68d48a8 Add changelog
• fcf956f Update example output
• 642c0d7 chore: Update ruff and uv
• 2a5ecb3 Replace instance_id by individual facets
• e2dcdfb docs: Changelog
• 85d4485 chore: add dirty and successful fields
• 624d1ac feat: Add filtering by facets to the executions list-groups
• 3a5fe94 chore: centralise console management
• 0f06b2b chore: Update regression outputs
• 1742005 chore: Handle more captions
• b4ce2b0 chore: Fix tests
• 8f42895 docs: Rest of changelog
• 85a9f37 docs: Changelog
• 86ed15a chore: rerun regression tests
• e08ebf3 chore: More clean ups
• 97a8424 feat: Update to include dimensions for the outputs
• 8627491 feat: Add dimensions to the CMEC format
• 279cdd1 chore: Fix tests
• 268d1d9 test: Add tests
• bb437db docs: Changelog
• 90ba329 chore: the _cv_dimension is required
• d7afee5 feat: Add dimension columns to the figure outputs
• 8959c8d changelog
• ad88acc ensure that volcello is associated with the run if needed
• 0c26d40 changed argument names weren't updated
• aee3064 Log ESMValTool recipe and config
• 522a2ee Add changelog
• 5099b36 Improve conversion from dataframe to recipe
• e91c97d Add changelog
• 37e3e06 Avoid selecting the areacella variable when filling the recipe
• 4f66d39 Re-order ESMValTool constraints
• 83e2fcc Use only apply style filters
• c4fb933 chore: Broad noqa to fix linting issue
• 4f03894 chore: Fix last test
• d5efa9e Rename some remaining metric occurrences to diagnostic
• 74c8ba7 Use logical or to combine multiple facet filters
• 03ac049 chore: don't include table id
• db44814 chore: Remove obs4REF data from the catalog
• 436a465 chore: Bump sample data to exclude the ta data
• 53997c3 chore: Remove the ta data that isn't needed any more.
• 0b56445 chore: Add more constraints
• 1654f0a chore: Update the ocean data
• 8985572 chore: Changelog
• e532605 Revert "Update number of files"
• 6e41089 Revert "Add ERA-5 obs4MIPs data to obs4REF"
• e50b691 docs: Update documentation
• 00f8c59 feat: Make the fetch script more flexible
• 6476797 feat: Make script more flexible
• f9a1b29 chore: Changelog
• 1251d75 fix: Gracefully handle empty directories
• f0dc00a chore: Changelog
• 3297a73 fix: Add PMP version to db
• 46e43f4 fix: Add PMP version to db
• ab406f6 Update example output
• d9f15f3 Improve ESMValTool series output
• 572ce23 docs: Changelog
• f9276a1 test: Add tests for database queries
• f77ef6b chore: Add logging to the ingest command
• 9ef2013 feat: Support adding/updating datasets
• 02db324 fix: no remove_emsembles for obs4mips
• fd40c67 chore: Add pmp lock file
• 46f283f chore: Include a statistic column
• 0922827 chore: Refactor out the series extraction to a separate file
• 385d378 fix: Added index and index_name into the db
• 70c6e0f Improve docstring
• f27f16c Improve docstring
• f065884 Use None as default value and add example to docstring
• 9af3e93 Apply more constraints to ESMValTool diagnostics
• 45722d5 Add groupby operation to require facets constraint
• 90519a4 Add constraint that areacello variables are available
• 1779166 Add changelog
• 5708c5f Specify the table_id for all recipes
• bdc269d Update example output but not pmp/annual-cycle
• 64b7bec Use "version" as the obs4MIPs version facet
• d310461 clean up
• e47dbb6 add change log
• 2073446 version update
• 6b30569 Update ESMValTool diagnostics to use timerange constraint
• 3b9fd47 Add a timerange constraint
• 8d30802 Add changelog
• 85de978 Update reference output
• 5e57968 Update sample data to v0.7.3
• dfef55c Add changelog
• 014401a Add test and rename existing tests
• 34a53ea Improve string representation of ExecutionDatasetCollection
• 99ded7a chore: Upgrade cmip6 dataset definitions
• 93e5e4a chore: Fix links in changelog

v0.6.6

Changelog

Features

• Added a diagnostic to create scatterplots of two cloud-relevant variables. (#261)

• Added support for serialising series metric values from an execution.
  These series metric values are also ingested into the database for later retrieval.

  Currently, only the ESMValTool example diagnostic supports this feature,
  but this will be extended to other diagnostics in the future. (#374)

• Added regional historical changes diagnostics. (#380)

• Shifts ILAMB executions to be per experiment/model/ensemble/grid as opposed to all models for a given experiment. Also dumps out series information into the database for all time traces found in the output files. (#391)

• Made it possible to run PMP diagnostics without having conda installed. (#392)

• Added climate drivers for fire diagnostic. (#393)

• Added a sub-command to ref executions that allows you to flag an execution as dirty. (#396)

Improvements

• Use a self-hosted CI runner for the integration tests.
  The GitHub runner do not have enough disk space to store the required datasets,
  and these data are downloaded over the internet on each run. (#365)
• Updated the sample data to v0.7.1. (#385)
• Adds additional indexes for some slow queries in the API (#395)
• Better error message when a user tries to use the HPCExecutor on Windows (#397)

Bug Fixes

• Fix bug in writing ESMValTool recipes that loses order of preprocessing steps.
  This bug was introduced in #378
  and included in the v0.6.5 release. (#384)

Changes

• 9064768 Bump version: 0.6.5 → 0.6.6
• b37c2ee fix: pip install statement for zsh
• d8b919c test: Add test for missing result
• 15d4caa docs: Changelog
• 20ad2ae doc: Update roadmap
• 37a633b chore: Handle HPCExector on Windows
• 77588fb chore: Ensure that the diagnostic could be pickled
• 03b67d8 changelog
• ed126f1 add test
• 280b9e6 restore setting the ilamb options
• 560fc05 added cli option to flag an execution as dirty
• c8862e8 update regression data
• a18622e expand ability to manually skip tests as needed
• 3ea1c12 data requirement changes
• 6a146b5 turns out I renamed this and forgot 🤦
• 530947c added an operator to RequireFacets that allow 'any' to be used in place of 'all' (default stays 'all')
• 9408f57 ilamb3 bump
• 2eac83c chore: Changelog
• 3937594 Switch fire diagnostic to pytest-regressions based test
• 83092f2 update tests
• c70df1a Update example diagnostic data to capture series output
• 999fb21 Remove unnessary code
• ea6a34e Better test name
• 762ace2 Replace recipe update tests by pytest-regressions
• a81a84b Update example output
• 27f38ff Record trend as metric
• d849eb6 Update example output
• 6ed57ec bump ilamb3
• d1d1031 changelog
• de2643c add a series per 1d trace in the ilamb output files
• 6062d14 Update ESMValTool version
• af2a5e9 Add and improve series output
• be699cc fix: Support upgrading
• a971b3c feat: Add extra database indexes
• 402444d Update provider test
• 7f34c9d Update test for 88c710f
• deb4e18 Add regression test data
• 331110d Skip regression tests for diagnostics with large output
• 025c11d Fix facets
• b1ff7fb Add test
• 5ba5fa8 Match new version of sample data
• 5fe5145 Normalize region names
• 88c710f Use obs4MIPs data from obs4REF and obs4MIPs directories
• d15c271 Add example output
• 7b4c8c6 Disable writing NetCDF output as it is ~300 MB
• c133ed6 Update sample data
• 05c2adf Add changelog
• bb34268 Add regional historical changes diagnostics
• 2dd39d8 Add changelog
• 1eee4a3 Add climate drivers for fire diagnostic
• 0edcb4e Fix test
• 07514d6 Only import climate_ref when type checking to avoid making it a dependency
• d99190b Add changelog
• b6fba8c Specify environment variables when running PMP diagnostic
• 786a03b shift to an execution per experiment/source/ensemble/grid, update bundles to include plots
• 08270ed Regenerate example output
• 1ebf803 Undo any changes to uv.lock
• 71a7928 Skip model with 360 day calendar
• 31582b8 Add changelog
• bb2c101 Mark comparison between datasets in database and as parsed as xfail because the database appears to only keep the latest version
• b2f6076 Update the number of expected metrics in solve test
• 4370165 Update test results
• 5a19e3a chore: Add test coverage
• a0a0a79 fix: Update path
• eff7d6f docs: Changelog
• 065163e docs: Fix docs
• 762e7d6 Update sample data to v0.7.1
• e6c068d Add changelog
• 7979e14 Fix bug in writing recipes that loses order of preprocessing steps
• 638ea46 chore: Ignore tests that include docker
• c0965b4 Update recipes hashes for newer ESMValTool version
• e01ff46 Update example output
• 934352f Update ESMValTool version
• 174f38d chore: Changelog
• 7419d7a test: Update tests to reflect new functionality
• 764faa3 feat: Serialise series data to disk
• 843216e chore: Mark docker as optional
• 450ebd6 chore: Symlink and don't fetch cache
• 4073017 fix: Use self-hosted runner for integration test
• 8b804a7 Demo use of series output
• 1348524 Update uv.lock
• ed0e9c9 Remove setuptools dependency, no idea how it got there
• a02b65a Fix recipe has and add regression test output
• 59bd7ba Move ruamel.yaml dependency to climate-ref-core
• af2b6b4 Use obs4MIPs sample data to populate obs4MIPs catalog
• a8ab6ad Add facets attribute
• 0bbf936 Improve test coverage
• b19261e Add new tests
• 6516013 Fix another test
• 3cc8659 Add changelog
• 11358c7 Update tests and rename metric slugs
• 48381ad Improve suptitle
• 5fcfe04 Add cloud scatterplots

v0.6.5

Announcements

Thanks to @rbeucher for his first contribution

Changelog

Features

• Add support for NCI PBS Scheduler to the HPCExecutor (#358)
• Added sea ice sensitivity diagnostic. (#378)
• Added data required for regional historical changes diagnostics. (#379)

Improvements

• Added a dataset ingestor for CMIP6 data that only uses the DRS information present in the filename. This greatly speeds up the process of ingesting a large number of files, as it does not require reading the file contents to extract the DRS information.

  This method can be opted into by specifying cmip6_parser: "drs" in the REF configuration file. The default parser remains the complete parser, which reads the file contents to all the required metadata, but this will change in a future PR. (#366)

Improved Documentation

• Address documentation review from the Model Benchmarking Task Team (#371)

Changes

• 6f4e698 Bump version: 0.6.4 → 0.6.5
• 66cc5df test: Fix test for select case
• 434adb9 chore: Fix typo in changelog
• effaa35 Apply suggestions from code review
• ab75cd7 Add ERA5 precipitation
• ace5645 Add changelog
• ce63ee9 Add data required for regional historical changes diagnostics
• 444cc8d Handle no match gracefully
• ab7ab49 Add test
• 8a50697 Update hash of ref/recipe_ref_cre.yml for newer ESMValTool version
• da001e3 Add changelog
• dc6fe1a Add regression test output
• 9ee3933 Fix omissions
• 9ed6a76 Add sea ice sensitivity diagnostic
• 0b721c0 final round
• 59b7691 more docs edits
• 4bf3a43 changelog
• 7db9aca address installation comments and attempt to remove confusion
• 302cdb3 edits post chat with Jitu
• 805a469 added packaging confusion
• 2b20f3c docs: Fix build process
• 1f6dcaf Add nodes_per_block parameter to SmartPBSProvider initialization
• 183b0c2 docs: Changelog
• d9f0056 chore: Move finalised to be a universal parameter
• 1300328 fix: Correct the integration test
• cb18c5e Remove redundant jobname entry from script generation test configuration
• 5468dc6 address some installation comments
• 0ffc558 spell out CLI the first use
• 3f9d2da remove 'bash' for blocks that print output to avoid syntax hylighting where it isn't wanted
• 0d47a6d chore: add finalised to the list of dataset attributes
• ee8cb1e test: Add tests for parser
• c3529a7 Fix precommit format issues
• 986b09c Fix pre-commit format issues
• b5bd171 chore: add a function to get the parsing function
• 2a5c33b chore: add a function to get the parsing function
• 8707fa6 Add test for SmartPBSProvider
• 82990d9 refactor: Add protocol for extracting
• 9ef2d0a Fix some logic
• 10f67b4 Fix name typo
• f29aca9 Modify docs
• a74630d Generalise PBSProvider
• 0898025 Add to changelog/ remove unused PBSProProvider import
• 8b55bcc Add support for NCI PBS Scheduler

v0.6.4

Changelog

Deprecations

• The --package option of the ref celery start-worker command has been deprecated and scheduled for removal.
  This functionality is now handled by the --provider option which uses entry points declared in the provider packages. (#367)

Features

• Use entrypoints to register provider plugins. (#360)
• Support celery workers to consume tasks for multiple providers (#367)

Improvements

• Add additional dimensions to the example metric for testing purposes. (#372)
• Added a basic script to the CMIP6 data targetted by the current set of diagnostics for the Assessment Fast Track. (#373)

Changes

• 176cf80 Bump version: 0.6.3 → 0.6.4
• 796dc61 chore: Fix test
• 54e902a docs: Changelog
• bde552c feat: Add script to download a large set of CMIP6 data for testing
• 0b14c9c docs: Changelog
• bf23fce fix: Increase the dimensionality of the example output
• 803298b chore: Add test for deprecation message
• 439ca6a feat: Support multiple providers
• 05fb4b0 chore: Declare provider as deprecated
• 63ae75c test: Update test suite
• 121d6fd chore: Better support for selecting providers in the celery worker
• 1a7411d Add test
• cb0742f Drop unique constraint on diagnostic slug
• 71f7949 Undo rename of example diagnostic
• 7aef2a7 Add changelog entry
• d14d0e1 Use entrypoints to register provider plugins

v0.6.3

Changelog

Improvements

• Use a new URL for serving the reference data.
  This should now support older versions of TLS which may help some users. (#364)

Changes

• 31b2a7e Bump version: 0.6.2 → 0.6.3
• cce2dfe docs: Changelog
• e917ad4 chore: Update download URL for datasets