Huge memory allocations

[Molkree]

I've just hit an unfortunate situation where OOIPy blows up RAM usage when getting one second of data.
Minimal reproducible example:

import datetime

from ooipy.request import hydrophone_request

start_time_str = "2021-09-19T12-16-49"
end_time_str = "2021-09-19T12-16-50"
node = "PC01A"

start_time = datetime.datetime.strptime(start_time_str, "%Y-%m-%dT%H-%M-%S")
end_time = datetime.datetime.strptime(end_time_str, "%Y-%m-%dT%H-%M-%S")

hydrophone_data = hydrophone_request.get_acoustic_data(
    start_time, end_time, node, verbose=True
)

In Google Colab I see ~10GB allocation and then one more ~10GB which crashes the kernel.

The same thing crashed our processing in GitHub VMs.

Here's why I narrowed it down to just one second of data:

[John-Ragland]

I think that this issue is caused by how data is organized from OOI's side of things. They have actually since fixed each mseed file to be 5 minutes (and they are doing some type of gap filling).

I'm closing this for now.