-
Notifications
You must be signed in to change notification settings - Fork 279
Initial dartsort wrapper implementation. #2125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
samuelgarcia
wants to merge
31
commits into
SpikeInterface:main
Choose a base branch
from
samuelgarcia:dartsort_wrapper
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
e1817f2
Initial dartsort wrapper implementation.
samuelgarcia e013256
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia 86800eb
wip
samuelgarcia c60b6a6
wip dartsort wrapper
samuelgarcia 2b76ead
output of darsort
samuelgarcia d445917
debug darsort output structure
samuelgarcia 5df983f
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia 186d133
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia a25835c
Improve backward compatibility for dartsort after the chunkable PR (4…
samuelgarcia a275086
Merge branch 'backward_compatibility_get_chunk_with_margin' of github…
samuelgarcia 2e679d0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] ed6c7e5
oups
samuelgarcia d8b658a
Merge branch 'backward_compatibility_get_chunk_with_margin' of github…
samuelgarcia 8de0656
oups
samuelgarcia 23a5775
Merge branch 'backward_compatibility_get_chunk_with_margin' of github…
samuelgarcia b5ee39a
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia 6e69d6c
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia 5c4ae84
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia a8424ac
Merge branch 'main' of github.com:SpikeInterface/spikeinterface into …
samuelgarcia 1e7953a
clean dartsort wrapper
samuelgarcia d1033c0
Update src/spikeinterface/sorters/external/dartsort.py
samuelgarcia 91612fe
Apply batched suggestions from code review
samuelgarcia d9c991f
Merge branch 'main' into dartsort_wrapper
samuelgarcia bc37b37
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] fb615ae
add dartsort docs
chrishalcrow ba4065c
Update src/spikeinterface/sorters/external/dartsort.py
chrishalcrow 04792b5
Merge branch 'main' into dartsort_wrapper
chrishalcrow ffeb1a5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] 669bec6
fix: use np.bincount for mode
alejoe91 bb333ef
fix: use sorting.save instead of write_sorting to preserve main_chann…
alejoe91 e071063
update dartsort refereces and docs
alejoe91 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| from pathlib import Path | ||
|
|
||
| import numpy as np | ||
|
|
||
| from ..basesorter import BaseSorter | ||
| from ...core import load | ||
|
|
||
|
|
||
| class DartsortSorter(BaseSorter): | ||
| """Dartsort wrapper""" | ||
|
|
||
| sorter_name = "dartsort" | ||
| requires_locations = False | ||
| compatible_with_parallel = {"loky": False, "multiprocessing": False, "threading": False} | ||
| sorter_description = """dartsort is a modular, drift-aware spike sorter developed in the Paninski lab. For installation and documentation, see https://dartsort.github.io""" | ||
| installation_mesg = """\nTo use dartsort run:\n | ||
| >>> pip install dartsort | ||
|
|
||
| More information about installing dartsort at: | ||
| * https://dartsort.github.io | ||
| """ | ||
|
|
||
| _default_params = {} | ||
|
|
||
| _params_description = {} | ||
|
|
||
| @classmethod | ||
| def _dynamic_params(cls): | ||
| from dartsort import DARTsortUserConfig | ||
| from pydantic import RootModel | ||
|
|
||
| # the trick is to transform the DARTsortUserConfig (a pydantic.dataclass) into a pydantic model | ||
| Model = RootModel[DARTsortUserConfig] | ||
| # so we can dump to dict | ||
| cfg = Model(DARTsortUserConfig()) | ||
| default_params = cfg.model_dump(mode="python") | ||
| # and retrieve properties | ||
| schema = Model.model_json_schema() | ||
| default_params_descriptions = {} | ||
| for k, props in schema["$defs"]["DARTsortUserConfig"]["properties"].items(): | ||
| default_params_descriptions[k] = props["title"] | ||
|
|
||
| return default_params, default_params_descriptions | ||
|
|
||
| @classmethod | ||
| def is_installed(cls): | ||
| try: | ||
| import dartsort | ||
|
|
||
| HAVE_DARTSORT = True | ||
| except ImportError: | ||
| HAVE_DARTSORT = False | ||
|
|
||
| return HAVE_DARTSORT | ||
|
|
||
| @staticmethod | ||
| def get_sorter_version(): | ||
| import dartsort | ||
|
|
||
| if hasattr(dartsort, "__version__"): | ||
| return dartsort.__version__ | ||
| return "unknown" | ||
|
|
||
| @classmethod | ||
| def _setup_recording(cls, recording, sorter_output_folder, params, verbose): | ||
| pass | ||
|
|
||
| @classmethod | ||
| def _run_from_folder(cls, sorter_output_folder, params, verbose): | ||
| from dartsort import dartsort as dartsort_main | ||
| from dartsort import DARTsortUserConfig | ||
|
|
||
| recording = cls.load_recording_from_folder(sorter_output_folder.parent, with_warnings=False) | ||
|
|
||
| # Dartsort can be given the motion object optionaly | ||
| motion = params.pop("motion", None) | ||
|
|
||
| # dartsort config are set using dataclass we need to map this | ||
| cfg = DARTsortUserConfig(**params) | ||
|
|
||
| ret = dartsort_main( | ||
| recording, | ||
| sorter_output_folder, | ||
| cfg, | ||
| motion=motion, | ||
| ) | ||
| # the DARTsortSorting is not the spikeinterface sorting | ||
| dartsort_sorting = ret["sorting"] | ||
| sorting = dartsort_sorting.to_numpy_sorting() | ||
|
|
||
|
chrishalcrow marked this conversation as resolved.
|
||
| # Add main_channel_id property by taking mode of channels from spikes | ||
| labels = dartsort_sorting.labels | ||
| spike_channels = dartsort_sorting.channels | ||
| main_channel_indices = [np.bincount(spike_channels[labels == unit_id]).argmax() for unit_id in sorting.unit_ids] | ||
| main_channel_ids = recording.channel_ids[main_channel_indices] | ||
| sorting.set_property("main_channel_id", main_channel_ids) | ||
| # We save to the final_darsort_sorting folder to propagate the main_channel_id property | ||
| sorting.save(folder=sorter_output_folder / "final_darsort_sorting") | ||
|
|
||
| @classmethod | ||
| def _get_result_from_folder(cls, sorter_output_folder): | ||
| sorter_output_folder = Path(sorter_output_folder) | ||
| sorting = load(sorter_output_folder / "final_darsort_sorting") | ||
| return sorting | ||
19 changes: 19 additions & 0 deletions
19
src/spikeinterface/sorters/external/tests/test_dartsort.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| import unittest | ||
| import pytest | ||
|
|
||
| from spikeinterface.sorters import DartsortSorter | ||
| from spikeinterface.sorters.tests.common_tests import SorterCommonTestSuite | ||
|
|
||
|
|
||
| @pytest.mark.skipif(not DartsortSorter.is_installed(), reason="dartsort not installed") | ||
| class DartsortCommonTestSuite(SorterCommonTestSuite, unittest.TestCase): | ||
| SorterClass = DartsortSorter | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| from pathlib import Path | ||
|
|
||
| test = DartsortCommonTestSuite() | ||
| test.cache_folder = Path(__file__).resolve().parents[4] / "cache_folder" / "sorters" | ||
| test.setUp() | ||
| test.test_with_run() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.