[Advice appreciated] How to implement pre-fetching?

[oaaij-gnahz]

Hi,

I am currently using SpikeInterface to preprocess my recording like shown below:

rec = si.read_nwb("recording.nwb")
rec = spre.band_pass(rec)
# a chain of SpikeInterface preprocessing functions ...

Recently I need to iterate over the entire recording for some custom processing function. Something like this:

results_by_chunk = [my_func(rec.get_traces(start_frame=beg, end_frame=beg+chunksize)) for beg in range(0, rec.get_num_frames(), chunksize)]

It seems like, the way I am using these functions, I need to wait for the disk IO and CPU-intensive processing functions sequentially and that is slow on my machine.

I would like to decouple the IO task and the CPU-intensive tasks. For example, one process for reading data chunks from disk to a queue, and another pool of processes to perform the preprocessing (the output will be way smaller in size than the timeseries data so they can all fit in memory). However I don't know how to start this under the lazy loading framework. How would you view this problem and do you have some suggestions?

Thank you very much!

[zm711]

If you're satisfied with our normal multiprocessing what you could do instead is make a custom preprocessing class by inheriting from BasePreprocessor and then you would just have your preprocessing added to the overall "recipe" of preprocessing lazily. The preprocessing would only be applied when needed and would be done in a chunk by chunk fashion like all of our other preprocessing. You could look at our filter preprocessors for inspiration here.

does that sound like what you want?

[h-mayorquin]

Maybe this would a use case for:
#2813

[oaaij-gnahz]

Hi Zach and Heberto,

Thank you so much for your reply! Your suggestions definitely make my work easier. Me not being familiar with the internals of SpikeInterface, though, I still have one question. One we run get_traces(), it sounds like the function will start from the first Recording class that is likely a file extractor, and blocks waiting for disk IO first, loads the data chunk into memory, and then perform preprocessing. All these happen sequentially, is that correct? If so the CPU may be idle when waiting for the data from disk. I wonder if there is some way we can speed this up by letting CPU preprocess an already loaded chunk simultaneously while the next chunk is being loaded. This use case probably is not relevant when user is only interested in one specific chunk of data; however, when iterating over the entire timeseries, I assume the parallelized IO and CPU preprocessing will greatly increase throughput. I don't have concrete evidence for how much speedup it will be, but this is what I have in mind when I try to optimize my code. What are your opinions?

Thanks again!

[zm711]

It sounds like you want to do something with async-await which allows you to fetch io and work on stuff with the CPU. I'm not super familiar with asynchronous programming but we don't specifically have that implementation set up. in this case you do need to load x chunks at a time, send them through the whole chain of preprocessing, then move onto the next set.

[oaaij-gnahz]

Oka, thanks for the pointers!

[alejoe91]

Hi Zach and Heberto,

Thank you so much for your reply! Your suggestions definitely make my work easier. Me not being familiar with the internals of SpikeInterface, though, I still have one question. One we run get_traces(), it sounds like the function will start from the first Recording class that is likely a file extractor, and blocks waiting for disk IO first, loads the data chunk into memory, and then perform preprocessing. All these happen sequentially, is that correct? If so the CPU may be idle when waiting for the data from disk. I wonder if there is some way we can speed this up by letting CPU preprocess an already loaded chunk simultaneously while the next chunk is being loaded. This use case probably is not relevant when user is only interested in one specific chunk of data; however, when iterating over the entire timeseries, I assume the parallelized IO and CPU preprocessing will greatly increase throughput. I don't have concrete evidence for how much speedup it will be, but this is what I have in mind when I try to optimize my code. What are your opinions?

Thanks again!

Let me comment here as well :)

Usually, when you want to get traces and do some operations (e.g., saving preprocessed to binary, extractong peaks, computing motion etc) we do this in parallel. Effectively, wach CPU will read a small portion of the data and apply the preprocessing chain. I don't think that using async calls would speed this up!

[oaaij-gnahz]

Hi Zach and Heberto,
Thank you so much for your reply! Your suggestions definitely make my work easier. Me not being familiar with the internals of SpikeInterface, though, I still have one question. One we run get_traces(), it sounds like the function will start from the first Recording class that is likely a file extractor, and blocks waiting for disk IO first, loads the data chunk into memory, and then perform preprocessing. All these happen sequentially, is that correct? If so the CPU may be idle when waiting for the data from disk. I wonder if there is some way we can speed this up by letting CPU preprocess an already loaded chunk simultaneously while the next chunk is being loaded. This use case probably is not relevant when user is only interested in one specific chunk of data; however, when iterating over the entire timeseries, I assume the parallelized IO and CPU preprocessing will greatly increase throughput. I don't have concrete evidence for how much speedup it will be, but this is what I have in mind when I try to optimize my code. What are your opinions?
Thanks again!

Let me comment here as well :)

Usually, when you want to get traces and do some operations (e.g., saving preprocessed to binary, extractong peaks, computing motion etc) we do this in parallel. Effectively, wach CPU will read a small portion of the data and apply the preprocessing chain. I don't think that using async calls would speed this up!

Thanks!