2025_hLSFM

Hyperspectral LSFM with DMD-only Shaping and Neural Network Reconstruction

We provide the code to reproduce the results reported in

Sébastien Crombez, Cédric Ray, Chloé Exbrayat-Héritier,Florence Ruggiero, Nicolas Ducros, "Hyperspectral LSFM with DMD-only Shaping and Neural Network Reconstruction," (2025).

• Main PDF
• Supplementary PDF

Contact: [email protected], CREATIS Laboratory, University of Lyon, France.

Installation

1. Create a conda environment

   conda create --name hsflsm
   conda activate hsflsm

2. Install pytorch using conda. E.g.,

   conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia
   conda install pip
   pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

   Visit https://pytorch.org/get-started/locally/ if you need a different installation.

3. Install SPyRiT v2.4.1 and a few more packages:

   pip install spyrit==2.4.1
   pip install spyder-kernels
   pip install opencv-python
   pip install pysptools
   pip install girder-client
   pip install natsort

opencv-python is required for spectral registration. It is not only necessary to run main_spectal_registration.py.

pysptools is required for spectral unmixing. It is not only necessary to run main_unmix_filter_EGFP-DsRed_14.py and main_unmix_filter_mRFP-DsRed.py.

natsoft is required to sort the files. It is not only necessary to run main_create_gif.py.

Code and data

1. Get source code and navigate to the /2025_hLSFM/ folder

   git clone https://github.com/openspyrit/spyrit-examples.git
   cd spyrit-examples/2025_hLSFM/ 

2. Download

• The raw measurements can be found here.
• The neural networks used for reconstruction can be found here. The covariance matrix used by the Tikhonov network can be found here.
• The reference spectra used for unmixing can be found here.

The directory structure should be

|---spyrit-examples
|   |---2025_hLSFM
|   |   |---data
|   |   |   |---2023_02_28_mRFP_DsRed_3D
|   |   |   |	|---Raw_data_chSPSIM_and_SPIM
|   |   |   |	|   |---data_2023_02_28
|   |   |   |	|   |   |---RUN0001
|   |   |   |	|   |   |---RUN0002
|   |   |   |	|   |   |---
|   |   |   |---2023_03_07_mRFP_DsRed_can_vs_had
|   |   |   |	|--- 
|   |   |   |---2023_03_13_2023_03_14_eGFP_DsRed_3D
|   |   |   |	|---
|   |   |   |---Reference_spectra
|   |   |---fonction
|   |   |---model
|   |   |   |---tikho-net_unet_imagenet_ph_50_exp_N_512_M_128_epo_20_lr_0.001_sss_10_sdr_0.5_bs_20_reg_1e-07.pth
|   |   |   |---
|   |   |---stat
|   |   |   |---Cov_1_512x512.npy
|   |   |---main_colorize_EGFP-DsRed_14.py
|   |   |---main_colorize_mRFP-DsRed.py
|   |   |---

Visualisation of the data

• Dataset 1 shows the raw measurements for one slice of the Tg(fli1:EGFP;olig2:DsRed) sample.
• Dataset 2 shows the abundance of EGFP (in green) and DsRed (in red) for all of the 25 slices of the Tg(fli1:EGFP;olig2:DsRed) sample. Many more visualizations of the abundance maps can be found here.
• Dataset 3 shows the conventional LSFM images for 21 slices of the Tg(fli1:EGFP;olig2:DsRed) sample. Other conventional LSFM images of the sample can be found here.
• Dataset 4 shows the abundance of mRFP (in cyan) and DsRed (in red) for all of the 20 slices of the Tg(sox10:mRFP;olig2:DsRed) sample. Many more visualizations of the abundance maps can be found here.
• Dataset 5 shows the conventional LSFM images for all slices of the Tg(sox10:mRFP;olig2:DsRed) sample. Other conventional LSFM images of the sample can be found here.

How to reproduce the results of the paper?

Figure 3: Fellgett's advantage and Tikhonov-Net

All the results are saved in .\data\2023_03_07_mRFP_DsRed_can_vs_had\. They can also be found on our warehouse (see here).

1. Run main_preprocess_Fellgett.py to generate the preprocessed measurements that will be saved in the subfolder .\Preprocess\
2. Run figure_3.py to reconstruct hypercubes that will be saved in the subfolder .\Reconstruction\hypercube\ and generate Fig. 3 of the paper.

Figure 4: EGFP-DsRed sample

All the results are saved in .\data\2023_03_13_2023_03_14_eGFP_DsRed_3D\. They can also be found on our warehouse (see here).

1. Run main_preprocess_EGFP-DsRed.py to generate the preprocessed measurements that will be saved in the subfolder .\Preprocess\
2. Run main_recon_net_EGFP-DsRed_14_all_slices.py to reconstruct the hypercubes that will be saved in the subfolder .\Reconstruction\hypercube\.
3. Run (the first sections of) main_spectal_registration.py to compensate for a spectral shift. The resulting hypercubes will be saved in the subfolder .\Reconstruction\hypercube\*_shift\.
4. Run main_unmix_filter_EGFP-DsRed_14.py to estimate the map of the different components (i.e., DsRed, EGFP, and autofluorescence) in the sample by both spectral filtering and spectral unmixing.
   • The maps obtained by spectral filtering will be saved in the subfolder .\Filtering_shift\.
   • The quantitative abundance maps obtained by spectral unmixing will be saved in the subfolder .\Unmixing_shift\.
5. Run main_visual_EGFP-DsRed_14.py to visualise the filter maps and quantitative maps in color. The resulting visualisations will be saved in the subfolder .\Visualisation_shift\.

Figure 5: DsRed-mRFP sample

All the results are saved in .\data\2023_02_28_mRFP_DsRed_3D\. They can also be found on our warehouse (see here).

1. Run main_preprocess_mRFP-DsRed.py to generate the preprocessed measurements that will be saved in the subfolder .\Preprocess\
2. Run main_recon_net_mRFP-DsRed_14_all_slices.py to reconstruct the hypercubes that will be saved in the subfolder .\Reconstruction\hypercube\.
3. Run (the last sections of) main_spectal_registration.py to compensate for a spectral shift. The resulting hypercubes will be saved in the subfolder .\Reconstruction\hypercube\*_shift\.
4. Run main_unmix_filter_EGFP-DsRed_14.py to estimate the map of the different components (i.e., DsRed, EGFP, and autofluorescence) in the sample by both spectral filtering and spectral unmixing.
   • The maps obtained by spectral filtering will be saved in the subfolder .\Filtering_shift\.
   • The quantitative abundance maps obtained by spectral unmixing will be saved in the subfolder .\Unmixing_shift\.
5. Run main_unmix_filter_mRFP-DsRed.py to visualise the filter maps and quantitative maps in color. The resulting visualisations will be saved in the subfolder .\Visualisation_shift\.

Miscellaneous

1. The script main_pattern_full_visu.py reproduces the illumination patterns of Fig. S6 of the supplementary document.
2. The scripts main_colorize_*.py plot in color ("rainbow colors")
   • the raw measurements in ./Preprocess/Run*/,
   • the reconstructed hypercubes in ./Reconstruction/hypercube/tikhonet50_div1.5/RUN*/.
3. The script main_stat.py shows examples to compute mean and covariance matrices "in 2D". To compute mean and covariance "in 1D", as required by the Tikhonov step of the reconstruction, see stat_1.py in spyrit.misc.statistics.
4. Training:
   • See train.sh for a typical shell script.
   • train.py is the main Python file (called in train.sh).
   • train_plot.py load and plot the training loss.