Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] - Add SpectralTimeModel and SpectralTimeEventModel #283

Merged
merged 110 commits into from
Mar 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
bb69219
initialize files / object for SpectralTimeModel
TomDonoghue Jul 4, 2023
2f6bda7
Merge branch 'name' into time
TomDonoghue Jul 9, 2023
3fb81a8
add sim_spectrogram
TomDonoghue Jul 10, 2023
edd6b3f
add plot_spectrogram
TomDonoghue Jul 10, 2023
8979a92
add data utils, including get_periodic_labels
TomDonoghue Jul 10, 2023
37156e4
add param over time template plots
TomDonoghue Jul 10, 2023
9fa9340
add plot_time_model func
TomDonoghue Jul 10, 2023
da2d657
add time model str gen func
TomDonoghue Jul 10, 2023
096c79f
add report gen & save for time model
TomDonoghue Jul 10, 2023
caa656a
plot updates related to time object
TomDonoghue Jul 10, 2023
0fa4d58
add get_results_by_ind
TomDonoghue Jul 10, 2023
a34fa6e
add SpectralTimeModel object
TomDonoghue Jul 10, 2023
2dd55cc
add tests for SpectralTimeModel
TomDonoghue Jul 10, 2023
7d1fb86
add tests for saving SpectralTimeModel
TomDonoghue Jul 10, 2023
5f1ed66
fix up time docs & add to init
TomDonoghue Jul 10, 2023
656532e
fix docs (group -> time)
TomDonoghue Jul 10, 2023
59eb3f4
tweak for knee params in time string
TomDonoghue Jul 10, 2023
8e60389
add explicit x-axis vals pass in to plot_params time plots
TomDonoghue Jul 10, 2023
258fb59
add data funcs
TomDonoghue Jul 10, 2023
ada4150
add plot_yshade template function
TomDonoghue Jul 10, 2023
77a82ff
fix up some time docstrings
TomDonoghue Jul 10, 2023
5f3b0ab
docstring cleans and moves
TomDonoghue Jul 10, 2023
5a28592
add plot_param_over_time_yshade
TomDonoghue Jul 10, 2023
a1a3e7f
update tests for data funcs
TomDonoghue Jul 10, 2023
62cd05c
use plot_yshade in spectra shade plot
TomDonoghue Jul 10, 2023
3d82e09
add str func for event model
TomDonoghue Jul 11, 2023
af7543b
add event object plot & test
TomDonoghue Jul 11, 2023
36ff83b
add get_results_by_row
TomDonoghue Jul 11, 2023
ebc7b20
update group object iter
TomDonoghue Jul 11, 2023
b7b71c5
misc small updates / fixes for time object
TomDonoghue Jul 11, 2023
8d2df8f
fix time reports save & test
TomDonoghue Jul 11, 2023
25fddab
add event object report save
TomDonoghue Jul 11, 2023
8f10c52
add SpectralTimeEventModel object
TomDonoghue Jul 11, 2023
f65aedb
add test for SpectralTimeEventModel
TomDonoghue Jul 11, 2023
57d4592
add SpectralTimeEventModel to inits
TomDonoghue Jul 11, 2023
b97efe2
test & fix event plot
TomDonoghue Jul 11, 2023
dd5b83b
lint cleanups
TomDonoghue Jul 11, 2023
21e90dc
add get_band_labels
TomDonoghue Jul 12, 2023
1d523ed
add band labels to model plots
TomDonoghue Jul 12, 2023
67d91bc
add get_model method to event object
TomDonoghue Jul 12, 2023
94a2272
add extract data utils & convs (include refactor out of event obj)
TomDonoghue Jul 12, 2023
3ce46c9
update time / event to_df & associated
TomDonoghue Jul 12, 2023
0c762b7
try actions tweak to run on all PRs
TomDonoghue Jul 13, 2023
164ef59
move min support to 3.6
TomDonoghue Jul 13, 2023
2a0402d
update n_peaks_ property
TomDonoghue Jul 13, 2023
ccd7c48
add tests for to_df
TomDonoghue Jul 13, 2023
da78eea
updates to save_model_report across objects
TomDonoghue Jul 13, 2023
ed8c31b
update naming & labels in data conversions
TomDonoghue Jul 13, 2023
feb8f82
move / refactor underlying funcs for get_params to own funcs
TomDonoghue Jul 13, 2023
a539ece
use new get_params funcs across all objs
TomDonoghue Jul 13, 2023
165b459
tweaks: clear time group_resulst & make time_results arrays
TomDonoghue Jul 14, 2023
91adcd0
update get_group method for time object
TomDonoghue Jul 14, 2023
0ab3562
event object accepts 3d arrays
TomDonoghue Jul 14, 2023
0cb32da
add get_group to event object and associated / related changes
TomDonoghue Jul 14, 2023
648e70e
add drop to time object
TomDonoghue Jul 14, 2023
9b133c2
add drop to event object and associated
TomDonoghue Jul 15, 2023
b947748
add plot_text helper plot function
TomDonoghue Jul 15, 2023
7a08faf
use plot_text in report generation
TomDonoghue Jul 15, 2023
ef4a909
make convert_results an explicitly public method
TomDonoghue Jul 15, 2023
6c3e11a
update arg input to spectrogram in time object
TomDonoghue Jul 17, 2023
4328513
add use of progress bar & parallelization to event object
TomDonoghue Jul 17, 2023
407bc4a
add get_files helper func
TomDonoghue Jul 17, 2023
8860aa4
fix select form get_files
TomDonoghue Jul 17, 2023
f34a59f
add save & load to event object
TomDonoghue Jul 17, 2023
3872e1b
refactor get_group, including add to get model with no inds
TomDonoghue Jul 17, 2023
551c870
use empty get_group
TomDonoghue Jul 17, 2023
457a250
apply same logic of upate to get_group to get_model
TomDonoghue Jul 17, 2023
0c832de
add length option to check_inds
TomDonoghue Jul 17, 2023
93b35b8
fix event get_group
TomDonoghue Jul 17, 2023
d972614
drop equivalent implementation event get_group subselection
TomDonoghue Jul 17, 2023
d2a48cb
add test for event parallel (& fix in doing so)
TomDonoghue Jul 17, 2023
bd81432
extend event get_group to support sub-object export
TomDonoghue Jul 17, 2023
addd59e
use new event get_group functionality
TomDonoghue Jul 17, 2023
409db0d
refactor event save to match others
TomDonoghue Jul 17, 2023
041067e
add tests & corresponding updates / fixes for event loading and assoc…
TomDonoghue Jul 17, 2023
aba8941
update plot_yshade to make avg / shade optional
TomDonoghue Jul 19, 2023
921a064
add shade options to pe & ap params
TomDonoghue Jul 19, 2023
14fbde0
fix specptrogram sim docstring
TomDonoghue Jul 20, 2023
d7e7165
Merge pull request #290 from fooof-tools/paramshade
TomDonoghue Jul 21, 2023
14aafbf
merge from name
TomDonoghue Jul 22, 2023
36efc12
fix / finish merge
TomDonoghue Jul 25, 2023
4c4802f
Merge branch 'name' into time
TomDonoghue Aug 3, 2023
f7fd489
Merge branch 'name' into time
TomDonoghue Aug 3, 2023
5bdb46e
Merge branch 'name' into time
TomDonoghue Aug 3, 2023
282d9e9
Merge branch 'name' into time
TomDonoghue Aug 3, 2023
97e45db
Merge branch 'name' into time
TomDonoghue Aug 3, 2023
9507039
merge name
TomDonoghue Sep 13, 2023
77a1fbb
fix merge
TomDonoghue Feb 15, 2024
c7375a7
plot_group -> plot_group_model (for consistency)
TomDonoghue Mar 26, 2024
9e7fb55
io updates, make consistent set of load funcs, and updates test
TomDonoghue Mar 26, 2024
5a561c7
update API for updates here
TomDonoghue Mar 26, 2024
62dba14
misc small doc fixes
TomDonoghue Mar 26, 2024
1a768e5
lints
TomDonoghue Mar 26, 2024
5f1d94a
add helper func to replace docstring params
TomDonoghue Mar 26, 2024
4f53829
finish lint: use docs updates & other small fixes
TomDonoghue Mar 26, 2024
3c049ff
small fix to periodic tests
TomDonoghue Mar 26, 2024
34db090
add get_band_peak_event function
TomDonoghue Mar 26, 2024
682c4a2
enforce consistent xlims on time & event param plots
TomDonoghue Mar 26, 2024
2698f8f
update event plots to plot even with nans
TomDonoghue Mar 26, 2024
21a1848
add compute_presence helper function
TomDonoghue Mar 26, 2024
0858a6c
refactor compute_presence
TomDonoghue Mar 26, 2024
6e641a4
add compute_arr_desc
TomDonoghue Mar 26, 2024
626479f
update string gen with new helpers
TomDonoghue Mar 26, 2024
e5161d0
tweak plots
TomDonoghue Mar 26, 2024
b6de612
tweak printing in event object
TomDonoghue Mar 27, 2024
508c0ba
add presence plot to event plots / reports
TomDonoghue Mar 27, 2024
0288fc4
add time tutorial
TomDonoghue Mar 27, 2024
c302af5
fix import
TomDonoghue Mar 27, 2024
2fa0aca
move tutorials for new addition
TomDonoghue Mar 27, 2024
ebba007
bump version number
TomDonoghue Mar 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,16 @@ on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
build:

# Tag ubuntu version to 20.04, in order to support python 3.6
# See issue: https://github.com/actions/setup-python/issues/544
# When ready to drop 3.6, can revert from 'ubuntu-20.04' -> 'ubuntu-latest'
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
env:
MODULE_NAME: specparam
strategy:
matrix:
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v3
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ This documentation includes:
Dependencies
------------

SpecParam is written in Python, and requires Python >= 3.6 to run.
SpecParam is written in Python, and requires Python >= 3.7 to run.

It has the following required dependencies:

Expand Down
38 changes: 34 additions & 4 deletions doc/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,17 @@ The SpectralGroupModel object allows for parameterizing groups of power spectra.

SpectralGroupModel

Time & Event Objects
~~~~~~~~~~~~~~~~~~~~

The time & event objects allows for parameterizing power spectra organized across time and/or events.

.. autosummary::
:toctree: generated/

SpectralTimeModel
SpectralTimeEventModel

Object Utilities
~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -155,6 +166,7 @@ The following functions take in model objects directly, which is the typical use

get_band_peak
get_band_peak_group
get_band_peak_event

**Array Inputs**

Expand All @@ -178,7 +190,7 @@ Code & utilities for simulating power spectra.
Generate Power Spectra
~~~~~~~~~~~~~~~~~~~~~~

Functions for simulating neural power spectra.
Functions for simulating neural power spectra and spectrograms.

.. currentmodule:: specparam.sim

Expand All @@ -187,6 +199,7 @@ Functions for simulating neural power spectra.

sim_power_spectrum
sim_group_power_spectra
sim_spectrogram

Manage Parameters
~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -242,14 +255,15 @@ Visualizations.
Plot Power Spectra
~~~~~~~~~~~~~~~~~~

Plots for visualizing power spectra.
Plots for visualizing power spectra and spectrograms.

.. currentmodule:: specparam.plts

.. autosummary::
:toctree: generated/

plot_spectra
plot_spectrogram

Plots for plotting power spectra with shaded regions.

Expand Down Expand Up @@ -311,7 +325,21 @@ Note that these are the same plotting functions that can be called from the mode
.. autosummary::
:toctree: generated/

plot_group
plot_group_model

.. currentmodule:: specparam.plts.time

.. autosummary::
:toctree: generated/

plot_time_model

.. currentmodule:: specparam.plts.event

.. autosummary::
:toctree: generated/

plot_event_model

Annotated Plots
~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -388,7 +416,9 @@ Input / Output (IO)
:toctree: generated/

load_model
load_group
load_group_model
load_time_model
load_event_model

Methods Reports
~~~~~~~~~~~~~~~
Expand Down
4 changes: 2 additions & 2 deletions examples/manage/plot_fit_models_3d.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
from specparam.sim import sim_group_power_spectra
from specparam.sim.utils import create_freqs
from specparam.sim.params import param_sampler
from specparam.utils.io import load_group
from specparam.utils.io import load_group_model

###################################################################################################
# Example Set-Up
Expand Down Expand Up @@ -229,7 +229,7 @@
###################################################################################################

# Reload our list of SpectralGroupModels
fgs = [load_group(file_name, file_path='results') \
fgs = [load_group_model(file_name, file_path='results') \
for file_name in os.listdir('results')]

###################################################################################################
Expand Down
1 change: 0 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
'Operating System :: POSIX',
'Operating System :: Unix',
'Programming Language :: Python',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
Expand Down
2 changes: 1 addition & 1 deletion specparam/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
from .version import __version__

from .bands import Bands
from .objs import SpectralModel, SpectralGroupModel
from .objs import SpectralModel, SpectralGroupModel, SpectralTimeModel, SpectralTimeEventModel
from .objs.utils import fit_models_3d
2 changes: 1 addition & 1 deletion specparam/analysis/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Analysis sub-module for model parameters and related metrics."""

from .error import compute_pointwise_error, compute_pointwise_error_group
from .periodic import get_band_peak, get_band_peak_group
from .periodic import get_band_peak, get_band_peak_group, get_band_peak_event
38 changes: 36 additions & 2 deletions specparam/analysis/periodic.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ def get_band_peak(model, band, select_highest=True, threshold=None,

Returns
-------
1d or 2d array
peaks : 1d or 2d array
Peak data. Each row is a peak, as [CF, PW, BW].

Examples
Expand Down Expand Up @@ -67,7 +67,7 @@ def get_band_peak_group(group, band, threshold=None, thresh_param='PW', attribut

Returns
-------
2d array
peaks : 2d array
Peak data. Each row is a peak, as [CF, PW, BW].
Each row represents an individual model from the input object.

Expand Down Expand Up @@ -101,6 +101,40 @@ def get_band_peak_group(group, band, threshold=None, thresh_param='PW', attribut
threshold, thresh_param)


def get_band_peak_event(event, band, threshold=None, thresh_param='PW', attribute='peak_params'):
"""Extract peaks from a band of interest from an event model object.

Parameters
----------
event : SpectralTimeEventModel
Object to extract peak data from.
band : tuple of (float, float)
Frequency range for the band of interest.
Defined as: (lower_frequency_bound, upper_frequency_bound).
select_highest : bool, optional, default: True
Whether to return single peak (if True) or all peaks within the range found (if False).
If True, returns the highest power peak within the search range.
threshold : float, optional
A minimum threshold value to apply.
thresh_param : {'PW', 'BW'}
Which parameter to threshold on. 'PW' is power and 'BW' is bandwidth.
attribute : {'peak_params', 'gaussian_params'}
Which attribute of peak data to extract data from.

Returns
-------
peaks : 3d array
Array of peak data, organized as [n_events, n_time_windows, n_peak_params].
"""

peaks = np.zeros([event.n_events, event.n_time_windows, 3])
for ind in range(event.n_events):
peaks[ind, :, :] = get_band_peak_group(\
event.get_group(ind, None, 'group'), band, threshold, thresh_param, attribute)

return peaks


def get_band_peak_group_arr(peak_params, band, n_fits, threshold=None, thresh_param='PW'):
"""Extract peaks within a given band of interest, from peaks from a group fit.

Expand Down
70 changes: 69 additions & 1 deletion specparam/core/io.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,32 @@ def fpath(file_path, file_name):
return full_path


def get_files(file_path, select=None):
"""Get a list of files from a directory.

Parameters
----------
file_path : Path or str
Name of the folder to get the list of files from.
select : str, optional
A search string to use to select files.

Returns
-------
list of str
A list of files.
"""

# Get list of available files, and drop hidden files
files = os.listdir(file_path)
files = [file for file in files if file[0] != '.']

if select:
files = [file for file in files if select in file]

return files


def save_model(model, file_name, file_path=None, append=False,
save_results=False, save_settings=False, save_data=False):
"""Save out data, results and/or settings from a model object into a JSON file.
Expand Down Expand Up @@ -130,7 +156,7 @@ def save_group(group, file_name, file_path=None, append=False,
file_name : str or FileObject
File to save data to.
file_path : Path or str, optional
Path to directory to load from. If None, loads from current directory.
Path to directory to load from. If None, saves to current directory.
append : bool, optional, default: False
Whether to append to an existing file, if available.
This option is only valid (and only used) if 'file_name' is a str.
Expand Down Expand Up @@ -168,6 +194,48 @@ def save_group(group, file_name, file_path=None, append=False,
raise ValueError("Save file not understood.")


def save_event(event, file_name, file_path=None, append=False,
save_results=False, save_settings=False, save_data=False):
"""Save out results and/or settings from event object. Saves out to a JSON file.

Parameters
----------
event : SpectralTimeEventModel
Object to save data from.
file_name : str or FileObject
File to save data to.
file_path : str, optional
Path to directory to load from. If None, saves to current directory.
append : bool, optional, default: False
Whether to append to an existing file, if available.
This option is only valid (and only used) if 'file_name' is a str.
save_results : bool, optional
Whether to save out model fit results.
save_settings : bool, optional
Whether to save out settings.
save_data : bool, optional
Whether to save out power spectra data.

Raises
------
ValueError
If the data or save file specified are not understood.
"""

fg = event.get_group(None, None, 'group')
if save_settings and not save_results and not save_data:
fg.save(file_name, file_path, append=append, save_settings=True)
else:
ndigits = len(str(len(event)))
for ind, gres in enumerate(event.event_group_results):
fg.group_results = gres
if save_data:
fg.power_spectra = event.spectrograms[ind, :, :].T
fg.save(file_name + '_{:0{ndigits}d}'.format(ind, ndigits=ndigits),
file_path=file_path, append=append, save_results=save_results,
save_settings=save_settings, save_data=save_data)


def load_json(file_name, file_path):
"""Load json file.

Expand Down
39 changes: 38 additions & 1 deletion specparam/core/modutils.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
"""Utility functions & decorators for the module."""

from importlib import import_module
from copy import deepcopy
from functools import wraps
from importlib import import_module

###################################################################################################
###################################################################################################
Expand Down Expand Up @@ -138,6 +139,42 @@ def docs_drop_param(docstring):
return front + back


def docs_replace_param(docstring, replace, new_param):
"""Replace a parameter description in a docstring.

Parameters
----------
docstring : str
Docstring to replace parameter description within.
replace : str
The name of the parameter to switch out.
new_param : str
The new parameter description to replace into the docstring.
This should be a string structured to be copied directly into the docstring.

Returns
-------
new_docstring : str
Update docstring, with parameter switched out.
"""

# Take a copy to make sure to avoid any potential aliasing
docstring = deepcopy(docstring)

# Find the index where the param to replace is
p_ind = docstring.find(replace)

# Find the second newline (end of to-replace param)
ti = docstring[p_ind:].find('\n')
n_ind = docstring[p_ind + ti + 1:].find('\n')
end_ind = p_ind + ti + 1 + n_ind

# Reconstitute docstring, replacing specified parameter
new_docstring = docstring[:p_ind] + new_param + docstring[end_ind:]

return new_docstring


def docs_append_to_section(docstring, section, add):
"""Append extra information to a specified section of a docstring.

Expand Down
Loading
Loading