Merge pull request #7 from pymc-devs/model-architecture

Add model architecture

Author: florence-bockting

.pre-commit-config.yaml

@@ -11,13 +11,14 @@ repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
     rev: "v6.0.0"
     hooks:
-      #- id: check-added-large-files
+      - id: check-added-large-files
       - id: check-ast
       - id: check-case-conflict
       - id: check-json
       - id: check-merge-conflict
       - id: check-symlinks
       - id: check-yaml
+        exclude: ^mkdocs\.yml$ # otherwise !!python/name:mermaid2.fence_mermaid in .mkdocs.yaml make it fail
       - id: debug-statements
       - id: detect-private-key
       - id: end-of-file-fixer

README.md

@@ -1,8 +1,5 @@
 <!--- --8<-- [start:description] -->
-# PyMC elicito
-
-Learning prior distributions for parameters in a Bayesian model based on expert information.
-
+# PyMC elicito: A Python package for expert prior elicitation using PyMC
 **Key info :**
 [![Docs](https://readthedocs.org/projects/pymc-elicito/badge/?version=latest)](https://pymc-elicito.readthedocs.io)
 [![Main branch: supported Python versions](https://img.shields.io/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Fpymc-devs%2Fpymc-elicito%2Fmain%2Fpyproject.toml)](https://github.com/pymc-devs/pymc-elicito/blob/main/pyproject.toml)
@@ -50,10 +47,13 @@ Some suggested options:
   and we won't reply to any issues
 -->
 
-**Prototype**: 
+**Prototype**:
 This project is just starting up and the code is all prototype.
 Pymc-elicito re-implements the Python package [`elicito`](https://github.com/florence-bockting/elicito) using pymc instead of tensorflow(-probability) as dependency.
 
+## Description
+**Expert prior elicitation** aims to define prior distributions for parameters within a Bayesian model that accurately incorporate the expectations of a domain expert. The `elicito` computational framework supports the modular implementation of diverse expert prior elicitation methods.
+
 <!--- --8<-- [end:description] -->
 
 Full documentation can be found at:

docs/NAVIGATION.md

@@ -9,6 +9,7 @@ See https://oprypin.github.io/mkdocs-literate-nav/
 - [Tutorials](tutorials/index.md)
 - [Further background](further-background/index.md)
     - [Dependency pinning and testing](further-background/dependency-pinning-and-testing.md)
+    - [Model architecture](further-background/model-architecture.md)
 - [Development](development.md)
 - [API reference](api/pymc_elicito/)
 - [Changelog](changelog.md)

docs/figures/conceptual-workflow.png

@@ -11,5 +11,4 @@ Points we will aim to cover:
 - Why it was created
 - Help the reader make connections
 
-We will aim to avoid writing instructions or technical descriptions here,
-they belong elsewhere.
+We will aim to avoid writing instructions or technical descriptions here, they belong elsewhere.

docs/further-background/index.md

@@ -0,0 +1,70 @@
+# Software Architecture of `elicito`
+The core computational workflow of the expert prior elicitation method implemented in `elicito` is based on a simulation-based optimization approach: Given a generative model and a set of initial hyperparameters defining the prior distributions, the model can be run in forward mode to simulate elicited summaries by computing the predefined target quantities and summary statistics. These simulated summaries are then compared with the
+expert-elicited summaries obtained during the expert-elicitation stage. An iterative optimization scheme is employed to update the hyperparameters of the parametric prior distributions so as to minimize the discrepancy between simulated and expert-elicited summaries. In other words, the objective is to identify the vector of hyperparameters that yields the closest alignment between simulated and expert-elicited summaries.
+
+```mermaid
+flowchart TB
+    subgraph eliobj["Elicit class (user input)"]
+            input["Model, Parameters, Targets, \n Network, Initializer, Optimizer, \n Meta-Setting"]
+            expert_dat["expert elicited summaries"]
+    end
+    subgraph initialization["initialization"]
+            initializer["initializer"]
+            hyper_parametric("prior parameters")
+            hyper_deep("weights/biases of DNNs")
+    end
+
+    subgraph subGraph3["priors"]
+            transformation["transform to constrained space"]
+            prior["sample from prior in \n unconstrained space"]
+    end
+    subgraph model["model"]
+            generative_model["run generative model \n in forward mode"]
+    end
+    subgraph summaries["summaries"]
+            elicits["elicited summaries"]
+            summary["target quantities"]
+    end
+    subgraph loss["loss"]
+            total_loss["total loss"]
+            indiv_loss["individual losses"]
+    end
+    subgraph optimization["optimization"]
+            gradients["gradients"]
+    end
+    subgraph training["training"]
+        subGraph3
+        prior_samples[/"prior samples"/]
+        model
+        model_output[/"model simulations"/]
+        summaries
+        simulated_summaries[/"simulated summaries"/]
+        loss
+        optimization
+        check{"convergence \n criterion"}
+        train_vars[/"trainable variables \n (=hyperparameters)"/]
+    end
+    input --> initializer
+    initializer -- if parametric prior --> hyper_parametric
+    initializer -- if deep prior --> hyper_deep
+    hyper_parametric --> train_vars
+    hyper_deep --> train_vars
+    train_vars --> prior
+    expert_dat --> indiv_loss & input
+    prior --> transformation
+    transformation --> prior_samples
+    prior_samples --> generative_model
+    generative_model --> model_output
+    model_output --> summary
+    summary --> elicits
+    elicits --> simulated_summaries
+    simulated_summaries --> indiv_loss
+    indiv_loss --> total_loss
+    total_loss --> gradients
+    gradients --> check
+    check -- not reached:<br> update --> train_vars
+    check -- reached --> stop((("stop")))
+
+    classDef data shape: lean-r
+    class prior_samples data
+```

docs/further-background/model-architecture.md

@@ -1 +1,61 @@
 ---8<--- "README.md:description"
+### Modular Capabilities of `elicito`
+Owing to its modular design, `elicito` accommodates key components across the entire elicitation workflow:
+
++ **Generative Models**: Support for a wide range of generative models (i.e., the statistical models describing the data-generating process).
++ **Expert Knowledge**: Flexibility in defining different types of expert knowledge (i.e., the specific information elicited from the domain expert).
++ **Elicitation Techniques**: Implementation of various types of elicitation techniques (e.g., quantile-based, histogram-based, or moment-based elicitation).
++ **Loss Functions**: Integration of loss functions (i.e., the criterion used to quantify the discrepancy between the expert knowledge and the simulated model quantities).
+
+### Computational Workflow
+The core logic of the expert prior elicitation method proposed in Bockting et al. (2024) can be summarized in a five-step workflow:
+
+/// note | Core logic of method underlying elicito
+1. *Define the generative model*: Specify the generative model, including the functional form
+of the data distribution and the parametric family of prior distributions.
+2. *Define target quantities and elicitation techniques*: Select the set of target quantities
+and determine the elicitation techniques to query the expert (cf. elicited summaries).
+3. *Simulate elicited summaries*: Draw samples from the generative model and compute the
+corresponding set of simulated elicited summaries.
+4. *Evaluate discrepancy between simulated and expert-elicited summaries*: Assess the
+discrepancy between the simulated and expert-elicited summaries using a multi-objective loss
+function.
+5. *Adjust prior hyperparameters to minimize discrepancy*: Apply an optimization scheme to
+update the prior hyperparameters such that the loss function is minimized.
+///
+
+![conceptual workflow](figures/conceptual-workflow.png)
+
+## Getting started
+ToDo
+
+### The `Elicit` class
+The primary user interface of **elicito** is the `Elicit` class, through which the user can specify the entire elicitation procedure. The arguments of the `Elicit` class are designed to capture all necessary information required to implement an elicitation method.
+A brief overview of these arguments is provided below:
+
++ `model`: Defines the generative model used in the elicitation procedure.
++ `parameters`: Specifies assumptions regarding the prior distributions over model parameters,
+    including (hyper)parameter constraints, dimensionality, and parametric form.
++ `targets`: Defines the elicited statistics in terms of target quantities and corresponding
+  elicitation techniques. Also specifies the discrepancy measure and weight used for the
+    associated loss component.
++ `expert`: Provides the expert information that serves as the basis for the learning criterion.
++ `optimizer`: Specifies the optimization algorithm to be used, along with its
+    hyperparameters (e.g., learning rate).
++ `trainer`: Configures the overall training procedure, including settings such as the random
+    seed, number of epochs, sample size, and batch size.
++ `initializer`: Defines the initialization strategy for the hyperparameters used to
+    instantiate the simulation-based optimization process.
++ `networks`: Specifies the architecture of the deep generative model; required only when
+    using non-parametric prior distributions.
+
+By configuring these core components, **elicito** supports a wide range of elicitation methods, including both structural and predictive approaches, univariate and multivariate as well as parametric and nonparametric prior distributions.
+
+## Main References
+
++ [Software Paper] Bockting F. & Bürkner PC (2025). elicito: A Python package for expert-prior elicitation. arXiv.
+[Preprint](https://arxiv.org/pdf/2506.16830)
++ [Methods Paper] Bockting F., Radev ST, Bürkner PC (2024). Simulation-based prior knowledge elicitation
+for parametric Bayesian models. *Scientific Report, 14*(1), 17330. [PDF](https://www.nature.com/articles/s41598-024-68090-7)
++ [Methods Paper] Bockting F., Radev ST, Bürkner PC (2025). Expert-elicitation method for non-parametric joint priors using
+normalizing flows. *Statistics and Computing*. [PDF](https://arxiv.org/abs/2411.15826)

docs/index.md

@@ -28,6 +28,7 @@ theme:
         name: Switch to light mode
 
 plugins:
+  - mermaid2
   # https://mkdocstrings.github.io/autorefs/
   - autorefs
   # Required for auto-generating our documentation stubs
@@ -121,6 +122,11 @@ plugins:
   - section-index
 
 markdown_extensions:
+  - pymdownx.superfences:
+      custom_fences:
+        - name: mermaid
+          class: mermaid
+          format: !!python/name:mermaid2.fence_mermaid
   # https://squidfunk.github.io/mkdocs-material/setup/extensions/python-markdown/#attribute-lists
   - attr_list
   - footnotes

mkdocs.yml

@@ -96,6 +96,7 @@ dev = [
 docs = [
     # Key dependencies
     # ----------------
+    "mkdocs-mermaid2-plugin>=1.2.3",
     "attrs==25.3.0",
     "mkdocs-autorefs==1.4.2",
     "mkdocs-gen-files==0.5.0",

pyproject.toml

@@ -25,6 +25,7 @@ cycler==0.12.1
 debugpy==1.8.11
 decorator==5.1.1
 defusedxml==0.7.1
+editorconfig==0.17.1
 exceptiongroup==1.3.0 ; python_full_version < '3.11'
 executing==2.1.0
 fastjsonschema==2.21.1
@@ -46,6 +47,7 @@ ipython-pygments-lexers==1.1.1 ; python_full_version >= '3.11'
 isoduration==20.11.0
 jedi==0.19.2
 jinja2==3.1.5
+jsbeautifier==1.15.4
 json5==0.10.0
 jsonpointer==3.0.0
 jsonschema==4.23.0
@@ -80,6 +82,7 @@ mkdocs-jupyter==0.25.1
 mkdocs-literate-nav==0.6.2
 mkdocs-material==9.6.16
 mkdocs-material-extensions==1.3.1
+mkdocs-mermaid2-plugin==1.2.3
 mkdocs-section-index==0.3.10
 mkdocstrings==0.30.0
 mkdocstrings-python==1.16.12