Fix reading of large multichannel Nihon Kohden EEG-1200A files#14060
Fix reading of large multichannel Nihon Kohden EEG-1200A files#14060mgj05otf wants to merge 2 commits into
Conversation
EEG-1200A systems store the real channel count, channel order, and the start-of-data offset in a chain of "extended blocks" rather than in the data block used by older NK systems. Falling back to the (essentially meaningless, for this format) data-block field caused large multichannel EEG-1200A recordings to be misread as a single ~1 second channel, a symptom reported by multiple users on the MNE forum and GitHub issues. By mgj05otf.
| buf[offset : offset + len(encoded)] = encoded | ||
|
|
||
|
|
||
| def _write_synthetic_1200a_eeg(tmp_path, ch_indices, sfreq, samples): |
There was a problem hiding this comment.
Rather than this, would it be possible to get a real data file from the system for example with 1s data and add it to mne-testing-data?
There was a problem hiding this comment.
Thank you for your review. I will look for non-clinical data that can be added to the test data. Please give me some moment.
|
To be good neighbors: reach out to Brainstorm (a GPL licensed project) and tell them we are reverse engineering this bit of code from them and ask permission? |
IIUC, Brainstorm reverse-engineered the structure of the binary file, and this PR just copies their implementation. So we definitely need permission to relicense the translated code from GPL to BSD. |
4 participants
Summary
mne.io.read_raw_nihonmisreads EEG-1200A recordings as a single ~1-secondchannel regardless of the file's actual size, channel count, or duration.
This is a known issue reported independently by multiple users:
Root cause
EEG-1200A systems store the real channel count, channel order, and the
start-of-data offset in a chain of "extended blocks" reached via a pointer
at file offset 0x03EE, rather than in the data block that older NK systems
(and the current MNE reader) use. That data-block field is essentially
meaningless for EEG-1200A files, so the reader was picking up junk values
(observed: 1 channel, 10 "duration units" -> 1 second) instead of the real
channel list and sample count.
The extended-block layout was reverse-engineered by the Brainstorm project;
this PR follows the same structure:
https://github.com/brainstorm-tools/brainstorm3/blob/master/toolbox/io/in_fopen_nk.m
Changes
mne/io/nihon/nihon.py: forEEG-1200A V01.00files, follow theextended-block chain to get the true channel count/order and data start
address; infer sample count from remaining file size (EEG-1200A doesn't
store a usable record count). Non-1200A files are unaffected. Files
claiming more than one control/data block for EEG-1200A now raise
NotImplementedErrorwith a clear message instead of silently misreading(this combination isn't understood yet, per the same limitation in
Brainstorm's reference implementation).
mne/io/nihon/tests/test_nihon.py: added a synthetic EEG-1200A fixture(no real patient data, randomly-generated samples) exercising the new
code path, plus a test for the multi-block error. Also fixed an unrelated
test-isolation bug in
test_nihon_duplicate_channels, which mutated theshared module-level
_default_chan_labelslist in place withoutrestoring it, corrupting state for tests run afterward in the same
session.
Testing
Verified against a real ~870 MB, 128-channel EEG-1200A clinical recording
(not included, for patient-privacy reasons): before this fix, MNE read it
as 1 channel / 1 second; after, it correctly reads 128 channels / 1684 s,
with lazy chunked reads matching a full preload exactly at arbitrary
offsets. Full existing test suite for this module still passes
(
test_nihon_eeg,test_nihon_calibrationcross-validate byte-for-byteagainst independent EDF exports of the same recordings).
AI assistance disclosure
Per CONTRIBUTING.md: this fix was developed with Claude Code assistance —
diagnosing the bug against a real recording, researching the EEG-1200A
extended-block format via the Brainstorm project's open-source
reverse-engineering, implementing the fix, and writing the synthetic test
fixture. I have reviewed and understand the changes.