Phlex v0.1.0

Important

This is a prototype release. It is known to have incomplete functionality with respect to the Phlex conceptual design; backwards compatibility of API is not guaranteed. The following API is intended for use:

• PHLEX_REGISTER_ALGORITHMS(...) opener for registering physics algorithms
• PHLEX_REGISTER_PROVIDERS(...) opener for registering data-product providers
• Anything in the top-level phlex namespace

Any interface (namespaces, functions, classes, etc.) that contains the strings experimental, internal, or detail should not be directly used.

---

Phlex v0.1.0 is the first prototype release of the Phlex framework. The purpose of the v0.1.0 release is to allow early adopters of Phlex to exercise its basic functionalities, which include:

• Specifying C++ algorithms as operators to the transform, observe, fold, and unfold higher-order functions
• Specifying Python algorithms as operators to the transform and observe higher-order functions
• Executing phlex programs using Jsonnet/JSON runtime configurations that load compiled plugins
• Configuring a Phlex-provided driver to produce a user-defined hierarchy (a tree) of data layers
• Writing data products with basic types to Phlex output files through FORM
• Accessing data-cell information through phlex::handle<T> objects for data products being used by algorithms

Instructions for building code that uses Phlex and running simple Phlex jobs can be found in the phlex-examples repository.

Specific Features

Phlex v0.1.0 provides the following capabilities:

1. API to register algorithms for C++ and Python
2. Phlex executable
3. Configuration 
   1. Supported language is Jsonnet
   2. Ability to configure which modules are loaded (includes ability to configure which algorithms specified in the module are registered)
   3. Ability to configure a Phlex-provided driver regarding which layers should be emitted
4. Product specification
   1. C++ language type
   2. Global product "name"
   3. Data layer
5. Higher-order functions
   1. Transform (C++, Python)
   2. Observer (C++, Python)
   3. Fold (C++)
   4. Unfold (C++)
6. Access to product information (including data-cell ID) through handles
7. Algorithms that can access multiple data products from the same data layer
8. Python algorithm that can consume a data product produced by a C++ algorithm
9. C++ algorithm that can consume a data product produced by a Python algorithm
10. C++ providers that can emit C++ data products

Included Pull Requests

• Optionally build with experimental FORM integration by @greenc-FNAL in #1
• Place form subdirectory at the top level by @knoepfel in #2
• Include formatting info for std::atomic<> by @greenc-FNAL in #3
• Space between quotes and suffix is deprecated in C++23 by @greenc-FNAL in #4
• Tweak CMake style to match Cetmodules by @greenc-FNAL in #6
• More spdlog/fmt fixes/improvements by @greenc-FNAL in #5
• Initial form prototype by @gemmeren in #8
• Remove dependence on boost backtrace by @wddgit in #12
• Remove set functions by @gemmeren in #13
• Form updates by @gemmeren in #11
• Ensure that ReadVector runs after WriteVector by @knoepfel in #23
• Continuous Integration on github Actions by @aolivier23 in #25
• Allow py:phlex to work without installing header files (use source instead) by @pcanal in #28
• Use configuration for I/O, plus general cleanup by @barnaliy in #22
• Apply clang-format 20 by @knoepfel in #26
• Warn only once per build directory about missing DOT by @pcanal in #30
• Ask github clangformat action's shell to not interpret comment by @pcanal in #31
• Remove for_each(...) registration API by @knoepfel in #34
• Adjust registration API for predicates and observers by @knoepfel in #35
• Remove need for detail::port_names(...) by @knoepfel in #37
• Use Boost PFR in node_catalog by @knoepfel in #38
• Change observer/predicate API to use input_family(...) by @knoepfel in #39
• Verify that a data product can be a vector of abstract types by @knoepfel in #40
• Use simple_ptr_map and other cleanups by @knoepfel in #41
• Apply clang-format to concept definitions by @knoepfel in #42
• Use different implementation for when(...) clause by @knoepfel in #43
• FORM container config by @aolivier23 in #36
• Workflow lint tweaks and rename for clarity by @greenc-FNAL in #47
• Actions for cmake-format by @greenc-FNAL in #48
• Turn reusable parts of cmake-build into composite actions by @greenc-FNAL in #51
• Add support and preliminary configuration for clang-tidy by @greenc-FNAL in #49
• Update CI container with dev tools by @greenc-FNAL in #55
• Refactoring to avoid unnecessary includes and minimize template instantiations by @knoepfel in #44
• Copilot instructions by @greenc-FNAL in #52
• Remove now-unnecessary inputs and conditional actions by @greenc-FNAL in #58
• Allow generator specification for actions by @greenc-FNAL in #56
• Add support and actions for coverage monitoring by @greenc-FNAL in #50
• clang-tidy-check should use composable actions to retain context by @greenc-FNAL in #59
• Fix coverage.sh path normalization by @greenc-FNAL in #64
• Add logic to bypass unneeded container load/workflow by @greenc-FNAL in #62
• Fix filtering race condition by @knoepfel in #65
• Resolve [clang-diagnostic-error] issues when running clang-tidy by @greenc-FNAL in #69
• Update and use image; fix/improve clang-tidy use by @greenc-FNAL in #77
• Disable CMake module scanning to prevent clang-diagnostic-error by @greenc-FNAL in #79
• Preliminary integration with Cetmodules 4 by @greenc-FNAL in #7
• Integrate clang-tidy with PR comments by @google-labs-jules[bot] in #80
• Improve coverage test reporting by @greenc-FNAL in #63
• Tweaks to clang-tidy configuration by @knoepfel in #82
• Create and schedule weekly CodeQL checks by @greenc-FNAL in #83
• Do not override clang-tidy header filter on command line by @knoepfel in #84
• [clang-tidy] Disable strict nodiscard and implicit Boolean conversion checks by @knoepfel in #88
• Update registration for transforms, folds, unfolds, and output modules by @knoepfel in #60
• maintenance/copilot instructions markdown by @greenc-FNAL in #89
• Adjust header guards to use LLVM style by @knoepfel in #93
• Remove unused code when forming graph by @knoepfel in #92
• Overhaul coverage tooling for GCC and Clang by @greenc-FNAL in #94
• Improvements to CI image, including installation of Ruff by @greenc-FNAL in #97
• Fix schema issue by @greenc-FNAL in #98
• Re-work image generation by @greenc-FNAL in #100
• Fix reference to removed image in coverage workflow by @greenc-FNAL in #103
• Adjust unfold so that it doesn't need a multiplexer by @knoepfel in #95
• Restore erroneously-removed sourcing of entrypoint.sh by @greenc-FNAL in #115
• Clang-tidy configuration adjustment by @knoepfel in #102
• Produce PR comment with summary on new or resolved alerts by @greenc-FNAL in #114
• Handles are valid upon construction by @knoepfel in #96
• Remove DOT file generation by @knoepfel in #116
• Fix "source" name issues for product_store by @greenc-FNAL in #121
• Make CodeQL PR comments actually useful by @greenc-FNAL in #117
• Fix issues reported by CodeQL by @greenc-FNAL in #104
• Trigger new CI runs on bot-generated commits by @greenc-FNAL in #123
• Reorganize and refactor workflows and actions by @greenc-FNAL in #129
• Trigger build on PR; add sanitizer builds by @greenc-FNAL in #124
• CI workflow to run Python ruff/mypy checks by @greenc-FNAL in #126
• Revert "CI workflow to run Python ruff/mypy checks" by @greenc-FNAL in #130
• Revert "Trigger build on PR; add sanitizer builds (#124)" by @greenc-FNAL in #131
• Resolve minor CodeQL Python reports by @greenc-FNAL in #134
• Resolve issue comment handling regressions by @greenc-FNAL in #135
• CI workflow to run Python ruff/mypy checks by @greenc-FNAL in #133
• Increase repo coverage threshold to 80% by @greenc-FNAL in #137
• Trigger build on PR; add sanitizer builds by @greenc-FNAL in #132
• Fix(actions): Make detect-relevant-changes robust to missing SHAs by @greenc-FNAL in #138
• feat: add actionlint workflow by @greenc-FNAL in #140
• Fix workflow issues caught by actionlint by @greenc-FNAL in #139
• Some FORM fixes by @knoepfel in #141
• Tighten library linking for FORM's test data products by @knoepfel in #142
• Add type checking to graph construction by @beojan in #119
• Reduce execution times for mock workflow by @knoepfel in #144
• Implement Trigger-Specific Build Matrix Defaults by @greenc-FNAL in #143
• Fix Python linting errors by @greenc-FNAL in #147
• Add CODEOWNERS: make @Framework-R-D/approvers code owners for the repo by @greenc-FNAL in #156
• Update terminology to agree with CDR by @knoepfel in #136
• Do not prematurely mark filter as ready for downstream processing by @knoepfel in #158
• Apply patches from #136 late review: fix header guard and rename levels_ to layers_ by @Copilot in #160
• Adjust more terminology in the code by @knoepfel in #162
• Make unfold interface reflect design document by @wddgit in #161
• Enable fork-based PRs to use fix workflows by @greenc-FNAL in #157
• Mandate the specification of the data layer in input_family(...) by @knoepfel in #155
• Add product_store::size() accessor by @knoepfel in #166
• Migrate library and installation configuration to use Cetmodules by @greenc-FNAL in #165
• Add missing TBB::tbb dependency for phlex_model library by @knoepfel in #167
• Fix Jsonnet Library Loading On MacOS by @marcpaterno in #170
• Convert FORM tests to use Cetmodules by @greenc-FNAL in #168
• Configure Dependabot for GitHub Actions updates by @greenc-FNAL in #171
• Fix bad commit hash for pinned action by @greenc-FNAL in #172
• Remove unused cached_product_stores class by @knoepfel in #176
• Bump github/codeql-action from 4.31.5 to 4.31.7 by @dependabot[bot] in #173
• Bump actions/setup-python from 6.0.0 to 6.1.0 by @dependabot[bot] in #175
• Bump actions/checkout from 6.0.0 to 6.0.1 by @dependabot[bot] in #174
• Bump codecov/codecov-action from 5.5.1 to 5.5.2 by @dependabot[bot] in #178
• Fix race condition in async_driver by @knoepfel in #181
• Providers by @sabasehrish in #180
• Remember to send flushes for unfolded data layers by @knoepfel in #183
• Enable direct conversion from configuration parameter to product_query by @knoepfel in #186
• Enable TSAN/ASAN compiler options for only GNU by @knoepfel in #191
• Bump github/codeql-action from 4.31.7 to 4.31.8 by @dependabot[bot] in #185
• Driver that generates layers according to configuration by @knoepfel in #179
• Bump actions/upload-artifact from 5.0.0 to 6.0.0 by @dependabot[bot] in #188
• Bump actions/download-artifact from 6.0.0 to 7.0.0 by @dependabot[bot] in #189
• Verify pipelining by @marcpaterno in #193
• Remove restriction on number of threads for framework_graph by @knoepfel in #194
• Bump github/codeql-action from 4.31.8 to 4.31.9 by @dependabot[bot] in #200
• Improve PR comments automatically generated from code formatting by @knoepfel in #202
• Fix stack-use-after-scope ASAN error by @knoepfel in #192
• Add FORM output module for phlex product_store integration by @barnaliy in #177
• Catch exceptions in main to ensure graceful exit and coverage data generation by @greenc-FNAL in #211
• Python support using converter nodes by @wlav in #45
• Add installation guide by @marcpaterno in #204
• Reorganize README and linked files by @marcpaterno in #215
• Make sure that plugins are installed by @knoepfel in #210
• Installation: do not recommend to delete the .spack directory by @pcanal in #216
• Support drivers and sources in job-configuration schema by @knoepfel in #195
• Specify plugin language as cpp or py by @knoepfel in #196
• Install one more header file by @knoepfel in #217
• updated instructions by @sabasehrish in #218
• Move core types from phlex::experimental to phlex namespace by @knoepfel in #206

New Contributors

• @greenc-FNAL made their first contribution in #1
• @knoepfel made their first contribution in #2
• @gemmeren made their first contribution in #8
• @wddgit made their first contribution in #12
• @aolivier23 made their first contribution in #25
• @pcanal made their first contribution in #28
• @barnaliy made their first contribution in #22
• @beojan made their first contribution in #119
• @marcpaterno made their first contribution in #170
• @sabasehrish made their first contribution in #180
• @wlav made their first contribution in #45

AI and Automated Contributions

• @google-labs-jules[bot] made their first contribution in #80
• @Copilot made their first contribution in #160
• @dependabot[bot] made their first contribution in #173

Full Changelog: https://github.com/Framework-R-D/phlex/commits/v0.1.0