v1.5.0

DataStreamCLI has undergone major new additions! Major changes include

• Automated LSTM BMI configuration file generation.
• Feature to convert troute output netcdf file to parquet for optimized data storage and reading.
• Implement the ability to create a routing-only datastreamcli execution wherein NWM channel routing files are used as input to NextGen troute. Simply provide the new configs/ngen/realization_sloth_troute.json as input at the cli and the forcingprocessor integration will handle the NWM channel routing conversion with its new added capability (https://github.com/CIROH-UA/forcingprocessor/releases/tag/v1.3.0)

The datastream container has been updated to

datastream-deps: "1.0.0"
datastream: "1.5.0"

The integration containers have been updated to

forcingprocessor: "2.0.1"
ciroh-ngen-image: "v1.7.0"
ngiab-teehr: "latest"

What's Changed

• add lstm bmi config gen feature with test by @JordanLaserGit in #23
• Update datastream version to 1.3.0 by @JordanLaserGit in #41
• Add manual workflow trigger with custom version tags by @harshavemula-ua in #43
• Disable push workflow trigger for pull requests by @harshavemula-ua in #29
• update options workflows by @JordanLaserGit in #38
• Refactor COPY commands in Dockerfile.datastream by @JordanLaserGit in #42
• default to docker versions set in version files by @JordanLaserGit in #54
• Fix VERBOSE flag and directory check bugs by @harshavemula-ua in #56
• Fix ARM workflow pre-cleanup to handle hidden directories by @harshavemula-ua in #57
• add python file for converting troute file to parquet by @JordanLaserGit in #40
• Consolidate CI/CD S3 test prefixes under test/cicd/datastream/ by @harshavemula-ua in #61
• Add hfsubset down note by @JordanLaserGit in #60
• Make LSTM folder consistent so generation can be skipped by @JordanLaserGit in #62
• merged routing_only_datastream and datastream script by @quinnylee in #51
• Update datastream version to 1.5.0 by @JordanLaserGit in #67
• Fix flag casing in ARM test execution template by @harshavemula-ua in #68

New Contributors

• @quinnylee made their first contribution in #51

Full Changelog: v1.3.0...v1.5.0

BMI Model Variable Fixes for v2.2 hydrofabric

This release focus on transformations to the v2.2 hydrofabric variables that are used in calculating the model variables contained within the BMI config files.

The associated docker containers have been updated to

datastream-deps: "1.0.1"
datastream: "1.2.1"

What's Changed

• fix hf2.2 unit conversions by @JoshCu in #34
• remove globals by @JordanLaserGit in #36
• scope basename into if to avoid basename on null by @JordanLaserGit in #28
• Update datastream versions in versions.yml by @JordanLaserGit in #37

New Contributors

• @JoshCu made their first contribution in #34

Full Changelog: v1.2.0...v1.3.0

Rust-based LSTM. v1.2.0

The recent development on DataStreamCLI resulted in the feature of being able to run the new Rust-based LSTM that was just included in the NGIAB build. DataStreamCLI will now generate the LSTM BMI configs and validate the Rust-base LSTM realization. This functionality uses the >v1.6.0 build of NGIAB (DS_TAG=v1.6.0).

Another change is DataStreamCLI now writes the t-route output files in the NRDS separately from the ngen-run tar file, accelerating data access of NRDS outputs.

What's Changed

• add test and configs for lstm by @JordanLaserGit in #18
• updated the workflows meant for DS by @harshavemula-ua in #21
• separate troute from tarball by @JordanLaserGit in #26

New Contributors

• @harshavemula-ua made their first contribution in #21

Full Changelog: v1.1.2...v1.2.0

Repo Split Complete, v1.1.2

What's Changed

All datastreamcli links in the docs have been updated to reflect the current repo and directory structure. So even workshops made previous to the split should be updated.

What's Next

Planned for future datastreamcli released are:

1. Enhanced testing strategies, #14.
2. Routing NWM outputs with troute, #15.
3. Ability to generate Rust-based lstm bmi config files, #17.

PR

• change links from ngen-datastream to datastreamcli by @JordanLaserGit in #16

Full Changelog: v1.1.1...v1.1.2

DataStreamCLI Repository Debut

This repository has been created from the DataStreamCLI module within the NextGen Research DataStream's repository ngen-datastream. All related tooling is planned to be removed from the ngen-datastream repository. Future development on DataStreamCLI will occur only in this repository.

Since the split, only directory structure changes have been made. The tooling remains unchanged.

What's Changed

• Reorganize dscli by @JordanLaserGit in #3

New Contributors

• @JordanLaserGit made their first contribution in #3

Full Changelog: https://github.com/CIROH-UA/datastreamcli/commits/v1.1.1