Skip to content

Commit 06f13a8

Browse files
satwikspssatwiksps
authored
docs: how-to guide for embedding time-series data (#1695)
* docs: add how-to guide for embedding time-series data using CausalCNN and Transformer (#1523) * docs: minor updates per review: explicit sampling, config comments * chore: fix Ruff and pre-commit formatting issues in time_series_embedding notebook
1 parent 0041a7a commit 06f13a8

File tree

2 files changed

+209
-0
lines changed

2 files changed

+209
-0
lines changed

‎docs/how_to_guide.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ Neural nets
3030

3131
how_to_guide/03_choose_neural_net.ipynb
3232
how_to_guide/04_embedding_networks.ipynb
33+
how_to_guide/20_time_series_embedding.ipynb
3334
how_to_guide/08_permutation_invariant_embeddings.ipynb
3435
how_to_guide/03_density_estimators.ipynb
3536

‎docs/how_to_guide/20_time_series_embedding.ipynb‎

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "42385334",
6+
"metadata": {},
7+
"source": [
8+
"# How to use time-series embeddings\n",
9+
"\n",
10+
"This short guide shows how to use **time-series embedding networks**:\n",
11+
"- `CausalCNNEmbedding`\n",
12+
"- `TransformerEmbedding`\n",
13+
"\n",
14+
"We’ll use a simple `torch.randn` simulator to demonstrate the workflow.\n",
15+
"\n",
16+
"> In real use, replace `torch.randn` with a dynamical simulator (e.g., SIR, Lotka–Volterra).\n"
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": null,
22+
"id": "e8a670dc",
23+
"metadata": {},
24+
"outputs": [],
25+
"source": [
26+
"import torch\n",
27+
"from torch import nn\n",
28+
"\n",
29+
"from sbi.inference import NPE\n",
30+
"from sbi.neural_nets import embedding_nets, posterior_nn\n",
31+
"from sbi.utils import BoxUniform"
32+
]
33+
},
34+
{
35+
"cell_type": "markdown",
36+
"id": "741c533f",
37+
"metadata": {},
38+
"source": [
39+
"Let's define a simple simulator that returns random time-series data to mimic sequential observations."
40+
]
41+
},
42+
{
43+
"cell_type": "code",
44+
"execution_count": null,
45+
"id": "230dd3cd",
46+
"metadata": {},
47+
"outputs": [],
48+
"source": [
49+
"torch.manual_seed(0)\n",
50+
"def simulator(theta):\n",
51+
" return torch.randn(1, 100)\n",
52+
"prior = BoxUniform(torch.tensor([0.0]), torch.tensor([1.0]))\n",
53+
"thetas = prior.sample((50,))\n",
54+
"xs = simulator(thetas)\n",
55+
"x_o = simulator(torch.tensor([0.5]))"
56+
]
57+
},
58+
{
59+
"cell_type": "markdown",
60+
"id": "021377e2",
61+
"metadata": {},
62+
"source": [
63+
"## Using CausalCNNEmbedding"
64+
]
65+
},
66+
{
67+
"cell_type": "markdown",
68+
"id": "9539c7dd",
69+
"metadata": {},
70+
"source": [
71+
"We use `CausalCNNEmbedding` to extract **local temporal patterns** from the sequence and train a Neural Posterior Estimator"
72+
]
73+
},
74+
{
75+
"cell_type": "code",
76+
"execution_count": null,
77+
"id": "981016ca",
78+
"metadata": {},
79+
"outputs": [],
80+
"source": [
81+
"# Define a causal CNN embedding for 1D time-series data\n",
82+
"embedding_cnn = embedding_nets.CausalCNNEmbedding(\n",
83+
" input_shape=(100,), # 1D time-series length\n",
84+
" num_conv_layers=3, # Number of CNN layers\n",
85+
" pool_kernel_size=10, # Pooling window for temporal downsampling\n",
86+
" output_dim=16, # Embedding size for NPE\n",
87+
")\n",
88+
"\n",
89+
"\n",
90+
"# Build density estimator with embedding\n",
91+
"density_estimator_cnn = posterior_nn(\n",
92+
" model=\"maf\",\n",
93+
" embedding_net=embedding_cnn,\n",
94+
" z_score_x=\"none\",\n",
95+
" z_score_y=\"none\",\n",
96+
")\n",
97+
"\n",
98+
"# Create and train NPE using the prior and simulated data\n",
99+
"inference_cnn = NPE(prior=prior, density_estimator=density_estimator_cnn)\n",
100+
"posterior_cnn = inference_cnn.append_simulations(thetas, xs).train()\n",
101+
"\n",
102+
"# Draw posterior samples given an observed time series\n",
103+
"samples_cnn = posterior_cnn.sample(torch.Size([10]), x_o)"
104+
]
105+
},
106+
{
107+
"cell_type": "markdown",
108+
"id": "24905e86",
109+
"metadata": {},
110+
"source": [
111+
"## Using TransformerEmbedding"
112+
]
113+
},
114+
{
115+
"cell_type": "markdown",
116+
"id": "5a6aca03",
117+
"metadata": {},
118+
"source": [
119+
"Next, we define a `TransformerEmbedding` that models **global dependencies** via self-attention."
120+
]
121+
},
122+
{
123+
"cell_type": "code",
124+
"execution_count": null,
125+
"id": "a030cab7",
126+
"metadata": {},
127+
"outputs": [],
128+
"source": [
129+
"# Transformer configuration for sequence embedding\n",
130+
"cfg = dict(\n",
131+
" vit=False, # Use standard transformer, not ViT-style\n",
132+
" feature_space_dim=192, # Internal embedding dimension (num_heads * head_dim)\n",
133+
" sequence_length=100, # Number of time points\n",
134+
" output_dim=16, # Output feature dimension for NPE\n",
135+
" num_layers=3, # Transformer depth\n",
136+
" num_heads=12, # Number of attention heads\n",
137+
" head_dim=16, # Size per attention head\n",
138+
" d_model=192, # ame as feature_space_dim\n",
139+
")\n",
140+
"\n",
141+
"# Initialize the transformer embedding network\n",
142+
"base_trans = embedding_nets.TransformerEmbedding(cfg)\n",
143+
"\n",
144+
"# Project 1D inputs to match transformer feature dimension\n",
145+
"class ProjectedTransformer(nn.Module):\n",
146+
" def __init__(self, transformer):\n",
147+
" super().__init__()\n",
148+
" self.proj, self.transformer = nn.Linear(1, 192), transformer\n",
149+
"\n",
150+
" def forward(self, x):\n",
151+
" if x.ndim == 2:\n",
152+
" x = x.unsqueeze(-1)\n",
153+
" x = self.proj(x)\n",
154+
" return self.transformer(x)\n",
155+
"\n",
156+
"embedding_trans = ProjectedTransformer(base_trans)\n",
157+
"\n",
158+
"# Build and train NPE with transformer embedding\n",
159+
"density_estimator_trans = posterior_nn(\n",
160+
" model=\"maf\",\n",
161+
" embedding_net=embedding_trans,\n",
162+
" z_score_x=\"none\",\n",
163+
" z_score_y=\"none\",\n",
164+
")\n",
165+
"\n",
166+
"inference_trans = NPE(prior=prior, density_estimator=density_estimator_trans)\n",
167+
"posterior_trans = inference_trans.append_simulations(thetas, xs).train()\n",
168+
"\n",
169+
"# Sample from the learned posterior\n",
170+
"samples_trans = posterior_trans.sample(torch.Size([10]), x_o)"
171+
]
172+
},
173+
{
174+
"cell_type": "markdown",
175+
"id": "7ee947ce",
176+
"metadata": {},
177+
"source": [
178+
"### Notes\n",
179+
"- **CausalCNNEmbedding** uses temporal convolutions for local dependencies. \n",
180+
"- **TransformerEmbedding** uses self-attention for global dependencies. \n",
181+
"- The small wrapper `ProjectedTransformer` projects scalar time-series to match the transformer’s feature space.\n",
182+
"- Both embeddings integrate seamlessly into `posterior_nn`.\n",
183+
"> Use GPU for long sequences or large networks.\n"
184+
]
185+
}
186+
],
187+
"metadata": {
188+
"kernelspec": {
189+
"display_name": ".venv (3.11.0)",
190+
"language": "python",
191+
"name": "python3"
192+
},
193+
"language_info": {
194+
"codemirror_mode": {
195+
"name": "ipython",
196+
"version": 3
197+
},
198+
"file_extension": ".py",
199+
"mimetype": "text/x-python",
200+
"name": "python",
201+
"nbconvert_exporter": "python",
202+
"pygments_lexer": "ipython3",
203+
"version": "3.11.0"
204+
}
205+
},
206+
"nbformat": 4,
207+
"nbformat_minor": 5
208+
}

0 commit comments

Comments
 (0)