solve conflicts

Author: alejoe91

.github/workflows/all-tests.yml

@@ -105,7 +105,7 @@ jobs:
         run: echo "dataset_hash=$(git ls-remote https://gin.g-node.org/NeuralEnsemble/ephy_testing_data.git HEAD | cut -f1)" >> $GITHUB_OUTPUT
 
       - name: Cache datasets
-        if: env.RUN_EXTRACTORS_TESTS == 'true'
+        if: env.RUN_EXTRACTORS_TESTS == 'true' || env.RUN_PREPROCESSING_TESTS == 'true'
         id: cache-datasets
         uses: actions/cache/restore@v4
         with:
@@ -115,7 +115,7 @@ jobs:
 
       - name: Install git-annex
         shell: bash
-        if: env.RUN_EXTRACTORS_TESTS == 'true'
+        if: env.RUN_EXTRACTORS_TESTS == 'true' || env.RUN_PREPROCESSING_TESTS == 'true'
         run: |
           pip install datalad-installer
           if [ ${{ runner.os }} = 'Linux' ]; then

.gitignore

@@ -141,7 +141,9 @@ examples/modules_gallery/**/*.zarr
 
 # Files and folders generated during tests
 test_folder/
+*trained_pipeline/
 
 # Mac OS
 .DS_Store
 test_data.json
+uv.lock

.pre-commit-config.yaml

@@ -6,7 +6,7 @@ repos:
     -   id: end-of-file-fixer
     -   id: trailing-whitespace
 -   repo: https://github.com/psf/black-pre-commit-mirror
-    rev: 26.1.0
+    rev: 26.3.1
     hooks:
     -   id: black
         files: ^src/

README.md

@@ -46,11 +46,18 @@
 
 [![Twitter](https://img.shields.io/badge/@spikeinterface-%231DA1F2.svg?style=for-the-badge&logo=Twitter&logoColor=white)](https://twitter.com/spikeinterface) [![Mastodon](https://img.shields.io/badge/-@spikeinterface-%232B90D9?style=for-the-badge&logo=mastodon&logoColor=white)](https://fosstodon.org/@spikeinterface)
 
+Please [Star](https://github.com/SpikeInterface/spikeinterface/stargazers) the project to support us and [Watch](https://github.com/SpikeInterface/spikeinterface/subscription) to always stay up-to-date!
 
-SpikeInterface is a Python framework designed to unify preexisting spike sorting technologies into a single code base.
+SpikeInterface is a Python package designed to unify preexisting spike sorting technologies into a single code base. If you use SpikeInterface, you are also using code and ideas from many other projects. Our codebase would be tiny without the amazing algorithms and formats that we interface with. See them all, and how to cite them, on our [references page](https://spikeinterface.readthedocs.io/en/latest/references.html). In the past year, we have added support for the following tools:
 
-Please [Star](https://github.com/SpikeInterface/spikeinterface/stargazers) the project to support us and [Watch](https://github.com/SpikeInterface/spikeinterface/subscription) to always stay up-to-date!
+- Bombcell [Bombcell: automated curation and cell classification of spike-sorted electrophysiology data](https://doi.org/10.5281/zenodo.8172822>) ([docs](https://spikeinterface.readthedocs.io/en/latest/how_to/auto_label_units.html#bombcell))
+- SLAy. [SLAy-ing oversplitting errors in high-density electrophysiology spike sorting](https://www.biorxiv.org/content/10.1101/2025.06.20.660590v2) ([docs](https://spikeinterface.readthedocs.io/en/latest/modules/curation.html#auto-merging-units))
+- Lupin, Spykingcicus2 and Tridesclous2. [Opening the black box: a modular approach to spike sorting](https://www.biorxiv.org/content/10.64898/2026.01.23.701239v1) ([docs](https://spikeinterface.readthedocs.io/en/stable/modules/sorters.html#supported-spike-sorters))
+- rtsort. [RT-Sort: An action potential propagation-based algorithm for real time spike detection and sorting with millisecond latencies](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0312438) ([docs](https://spikeinterface.readthedocs.io/en/stable/modules/sorters.html#supported-spike-sorters))
+- MEDiCINe. [MEDiCINe: Motion Correction for Neural Electrophysiology Recordings](https://www.eneuro.org/content/12/3/ENEURO.0529-24.2025) ([docs](https://spikeinterface.readthedocs.io/en/latest/how_to/handle_drift.html))
+- UnitRefine. [UnitRefine: A Community Toolbox for Automated Spike Sorting Curation](https://www.biorxiv.org/content/10.1101/2025.03.30.645770v2) ([docs](https://spikeinterface.readthedocs.io/en/latest/tutorials_custom_index.html#automated-curation-tutorials))
 
+If you would like us to add another tool, or you would like to integrate your project with our package, please [open an issue](https://github.com/SpikeInterface/spikeinterface/issues).
 
 With SpikeInterface, users can:
 

doc/api.rst

@@ -8,10 +8,33 @@ spikeinterface.core
     .. autofunction:: load
     .. autoclass:: BaseRecording
         :members:
+    .. automethod:: BaseRecording.save
+    .. automethod:: BaseRecording.save_to_memory
+    .. automethod:: BaseRecording.save_to_folder
+    .. automethod:: BaseRecording.save_to_zarr
+    .. automethod:: BaseRecording.dump
+    .. automethod:: BaseRecording.dump_to_json
+    .. automethod:: BaseRecording.dump_to_pickle
+    .. automethod:: BaseRecording.remove_channels
     .. autoclass:: BaseSorting
         :members:
+    .. automethod:: BaseSorting.save
+    .. automethod:: BaseSorting.save_to_memory
+    .. automethod:: BaseSorting.save_to_folder
+    .. automethod:: BaseSorting.save_to_zarr
+    .. automethod:: BaseSorting.dump
+    .. automethod:: BaseSorting.dump_to_json
+    .. automethod:: BaseSorting.dump_to_pickle
     .. autoclass:: BaseSnippets
         :members:
+    .. automethod:: BaseSnippets.save
+    .. automethod:: BaseSnippets.save_to_memory
+    .. automethod:: BaseSnippets.save_to_folder
+    .. automethod:: BaseSnippets.save_to_zarr
+    .. automethod:: BaseSnippets.dump
+    .. automethod:: BaseSnippets.dump_to_json
+    .. automethod:: BaseSnippets.dump_to_pickle
+    .. automethod:: BaseSnippets.remove_channels
     .. autoclass:: BaseEvent
         :members:
     .. autoclass:: SortingAnalyzer
@@ -204,6 +227,7 @@ spikeinterface.preprocessing
     .. autofunction:: remove_artifacts
     .. autofunction:: resample
     .. autofunction:: scale
+    .. autofunction:: scale_to_uV
     .. autofunction:: silence_periods
     .. autofunction:: unsigned_to_signed
     .. autofunction:: whiten
@@ -287,8 +311,6 @@ spikeinterface.comparison
     .. autofunction:: compare_sorter_to_ground_truth
     .. autofunction:: compare_templates
     .. autofunction:: compare_multiple_templates
-    .. autofunction:: create_hybrid_units_recording
-    .. autofunction:: create_hybrid_spikes_recording
 
     .. autoclass:: GroundTruthComparison
         :members:
@@ -375,6 +397,8 @@ spikeinterface.curation
     .. autofunction:: remove_duplicated_spikes
     .. autofunction:: remove_excess_spikes
     .. autofunction:: threshold_metrics_label_units
+    .. autofunction:: bombcell_label_units
+    .. autofunction:: bombcell_get_default_thresholds
     .. autofunction:: model_based_label_units
     .. autofunction:: load_model
     .. autofunction:: train_model
@@ -388,7 +412,7 @@ such as merging and splitting units, as well as defining labels for units.
 
 .. automodule:: spikeinterface.curation.curation_model
 
-    .. autopydantic_model:: CurationModel
+    .. autopydantic_model:: Curation
     .. autopydantic_model:: Merge
     .. autopydantic_model:: Split
     .. autopydantic_model:: ManualLabel

doc/development/development.rst

@@ -248,6 +248,74 @@ Note, however, that the running time of the command above will be quite long. If
 
     pytest src/spikeinterface/core/ --cov=spikeinterface/core --cov-report html
 
+
+Integrate your software with SpikeInterface
+-------------------------------------------
+
+You've made a great analysis tool for ephys data - congrats! It's popular and people are using it.
+Why would you integrate it into SpikeInterface? And how can do you do that?
+
+Why integrate it
+^^^^^^^^^^^^^^^^
+
+SpikeInterface is designed to allow users to 1) read in any recording format 2) apply a wide variety
+of preprocessing steps to the recording 3) run any sorter on the data 4) compute standard
+postprocessing data about the sorting (and more!!). We work hard to be general and allow users to do
+anything they'd like with their data. Hence if your tool works with SpikeInterface, it works with an
+enormous array of possible analysis pipelines.
+
+When they're first written, analysis tools are usually designed in a specific lab and are tested
+with a small number of collaborators and datasets. It's technically difficult to design a tool
+which can be applied to data from a wide variety of devices, set-ups and sorters. There are many
+edge cases and "gotcha"s to consider. But the SpikeInterface team have extensive experience with
+exactly this problem. We know how to write code so that it doesn't depend on data-specific or
+sorter-specific quirks. Hence we can help generalize the tool and widen its potential userbase.
+
+Often, the first step when using an analysis tool is to wrangle the data into a specific format:
+"please save your recording to a binary file which is preprocessed in a certain way" or "ensure your
+sorting output is in this format". SpikeInterface deals with this painful data wrangling for you: we
+load many formats and internally represent them in a consistent way. We also apply all preprocessing
+steps lazily, meaning that users never need to save a copy of a preprocessed recording: they can work
+directly from the raw recording. We believe this reduces the barrier to entry for the end user, which
+makes your tool more likely to be used by many researchers.
+
+How to integrate it
+^^^^^^^^^^^^^^^^^^^
+
+If you'd like to integrate your tool into SpikeInterface: amazing! Just
+`raise an issue <https://github.com/SpikeInterface/spikeinterface/issues>`_. There are two main ways
+to integrate your software into SpikeInterface.
+
+1) Wrapping
+
+You keep your code in your own codebase, and write a thin wrapper in SpikeInterface which interacts
+with your library. Most external sorters that SpikeInterface can call are organized in this way. To
+use the tool, the user will have to install your package and the maintenance of the software is primarily
+on your shoulders.
+
+2) Integration
+
+You re-implement your code and add it to the SpikeInterface package. Many preprocessing steps, motion
+correction algorithms and postprocessing steps are implemented like this. Your method and code become
+part of the SpikeInterface package and we take on the maintenance burden (with your help, we hope!).
+
+In this case, we don't expect the re-implementation to be a perfect copy of the code, since we may need
+to change the code to work for a more general set of data.
+
+
+Credit and Citation
+^^^^^^^^^^^^^^^^^^^
+
+We want to ensure the tools that SpikeInterface wrap and integrate are properly cited. We'll ensure that
+credit is given when the tool is mentioned in the docs, in the docstring of the function, and we will
+add it to the :doc:`../references` page. When added, we will include your software on the front-page of
+the SpikeInterface GitHub for at least 1 year. If we fail to cite you properly, this is certainly a mistake.
+Please `raise an issue <https://github.com/SpikeInterface/spikeinterface/issues>`_.
+
+See all the packages and projects which have been wrapped or integrated into SpikeInterface in our
+:doc:`../references` page.
+
+
 Implement a new extractor
 -------------------------
 

doc/get_started/quickstart.rst

@@ -247,7 +247,7 @@ object to disk.
 
 
 To reload a preprocessed recording that was saved to disk, you can use
-``load_extractor()`` function from the ``core`` module.
+``load()`` function from the ``core`` module.
 
 Now you are ready to spike sort using the ``spikeinterface.sorters``
 module! Let’s first check which sorters are implemented and which are