Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for embedding images in .pkg.slp #91

Merged
merged 30 commits into from
Jun 5, 2024
Merged

Add support for embedding images in .pkg.slp #91

Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
18c06e3
Add serialization to dict at the video object level
talmo May 5, 2024
6bc26c4
Store and retrieve video backend metadata
talmo May 5, 2024
84f9542
Try to restore source video when available
talmo May 5, 2024
a15f44b
Rename symbol
talmo May 5, 2024
2dbaea0
Use backend metadata when available when serializing
talmo May 5, 2024
d933cc0
Fix backend metadata factory
talmo May 5, 2024
247f92e
Re-embed videos when saving labels with embedded videos
talmo May 5, 2024
d35f07e
Fix serialization and logic for checking for embedded images
talmo May 5, 2024
31341da
Fix multi-frame decoding
talmo May 5, 2024
4402cb4
Fix docstring order
talmo May 5, 2024
696ae60
Add method to embed a list of frames and update the objects
talmo May 5, 2024
fd9bae9
Fix order of operations
talmo May 5, 2024
b41aa38
Add embed_videos
talmo May 6, 2024
c8c5ebb
Fix mid-level embedding function
talmo May 17, 2024
5f70e1c
Hash videos by ID
talmo May 17, 2024
8fe51c0
Add property to return embedded frame indices
talmo May 17, 2024
c329bad
Hash LabeledFrame by ID and add convenience checks for instance types
talmo May 17, 2024
527de4e
Labels.user_labeled_frames
talmo May 17, 2024
77f7ef4
Fix JABS
talmo May 17, 2024
b71646f
Tests
talmo May 17, 2024
4e5734b
Add live coverage support
talmo May 17, 2024
0614541
Expose high level embedding
talmo Jun 5, 2024
3894837
Separate replace video and support restoring source
talmo Jun 5, 2024
a1864a6
Merge branch 'main' into talmo/embed-pkg-slp
talmo Jun 5, 2024
a76680a
Lint
talmo Jun 5, 2024
64f88e5
Add Video(filename) syntactic sugar
talmo Jun 5, 2024
8932247
Coverage
talmo Jun 5, 2024
eff9676
Windows test fix
talmo Jun 5, 2024
5eee083
Windows test fix again
talmo Jun 5, 2024
d2a1415
Fix test
talmo Jun 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ coverage.xml
*.py,cover
.hypothesis/
.pytest_cache/
lcov.info

# Translations
*.mo
Expand Down
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,15 @@ We check for coverage by parsing the outputs from `pytest` and uploading to [Cod

All changes should aim to increase or maintain test coverage.

### Live coverage

*The following steps are based on [this guide](https://jasonstitt.com/perfect-python-live-test-coverage).*

1. If you already have an environment installed, `pip install -e ."[dev]"` to make sure you have the latest dev tools (namely `pytest-watch`).
2. Install the [Coverage Gutters extension](https://marketplace.visualstudio.com/items?itemName=ryanluker.vscode-coverage-gutters) in VS Code.
3. Open a terminal, `conda activate sleap-io` and then run `ptw` to automatically run tests. This will generate a new `lcov.info` file when it's done.
4. Enable the coverage gutters by using **Ctrl/Cmd**+**Shift**+**P**, then **Coverage Gutters: Display Coverage**.


### Code style
To standardize formatting conventions, we use [`black`](https://black.readthedocs.io/en/stable/).
Expand Down
24 changes: 15 additions & 9 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ name = "sleap-io"
authors = [
{name = "Liezl Maree", email = "[email protected]"},
{name = "David Samy", email = "[email protected]"},
{name = "Talmo Pereira", email = "[email protected]"}
]
{name = "Talmo Pereira", email = "[email protected]"}]
description="Standalone utilities for working with pose data from SLEAP and other tools."
requires-python = ">=3.7"
keywords = ["sleap", "pose tracking", "pose estimation", "behavior"]
Expand All @@ -19,8 +18,7 @@ classifiers = [
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12"
]
"Programming Language :: Python :: 3.12"]
dependencies = [
"numpy",
"attrs",
Expand All @@ -31,8 +29,7 @@ dependencies = [
"simplejson",
"imageio",
"imageio-ffmpeg",
"av"
]
"av"]
dynamic = ["version", "readme"]

[tool.setuptools.dynamic]
Expand All @@ -43,6 +40,7 @@ readme = {file = ["README.md"], content-type="text/markdown"}
dev = [
"pytest",
"pytest-cov",
"pytest-watch",
"black",
"pydocstyle",
"toml",
Expand All @@ -52,16 +50,24 @@ dev = [
"mkdocs-jupyter",
"mkdocstrings[python]>=0.18",
"mkdocs-gen-files",
"mkdocs-literate-nav"
]
"mkdocs-literate-nav"]

[project.urls]
Homepage = "https://sleap.ai"
Homepage = "https://io.sleap.ai"
Repository = "https://github.com/talmolab/sleap-io"

[tool.setuptools.packages.find]
exclude = ["site"]

[tool.black]
line-length = 88

[pydocstyle]
convention = "google"
match-dir = "sleap_io"

[tool.coverage.run]
source = ["livecov"]

[tool.pytest.ini_options]
addopts = "--cov sleap_io --cov-report=lcov:lcov.info --cov-report=term"
3 changes: 2 additions & 1 deletion sleap_io/io/jabs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ def read_labels(
frames: List[LabeledFrame] = []
# Video name is the pose file minus the suffix
video_name = re.sub(r"(_pose_est_v[2-6])?\.h5", ".avi", labels_path)
video = Video.from_filename(video_name)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Several instances of read_labels are still passing filenames instead of Video objects. These need to be updated to align with the new implementation.

  • tests/io/test_slp.py

    • labels = read_labels(slp_typical)
    • labels = read_labels(slp_simple_skel)
    • labels = read_labels(slp_minimal)
    • labels = read_labels(slp_predictions_with_provenance)
    • labels = read_labels(slp_real_data)
    • labels = read_labels(centered_pair)
    • labels = read_labels(fn)
    • labels = read_labels(tmp_path / "test.slp")
    • labels = read_labels(slp_imgvideo)
    • labels = read_labels(tmpdir / "test.slp")
    • labels = read_labels(slp_minimal_pkg)
    • labels = read_labels(str(tmpdir / "roundtrip.pkg.slp"))
    • labels = read_labels(labels_path)

  • tests/io/test_labelstudio.py

    • labels = slp_read_labels(slp_typical)
    • labels = slp_read_labels(slp_simple_skel)
    • labels = slp_read_labels(slp_minimal)
    • ls_labels = read_labels(file_path, skeleton)

  • tests/io/test_jabs.py

    • slp_labels = read_labels(jabs_real_data_v5)
    • labels = read_labels(jabs_real_data_v5)

  • sleap_io/io/main.py

    • return slp.read_labels(filename)
    • return labelstudio.read_labels(filename, skeleton=skeleton)
    • return jabs.read_labels(filename, skeleton=skeleton)

Please update these instances to pass Video objects instead of filenames.

Analysis chain

Use of existing Video object in read_labels enhances efficiency and code reuse.

Scripts Executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all function calls to `read_labels` pass a `Video` object.

# Test: Search for the function usage. Expect: Only occurances with a `Video` object.
rg --type python $'read_labels'

Length of output: 61

Script:

#!/bin/bash
# Description: Verify all function calls to `read_labels` pass a `Video` object.

# Test: Search for the function usage. Expect: Only occurrences with a `Video` object.
rg 'read_labels' --glob '*.py'

Length of output: 2566

Script:

#!/bin/bash
# Description: Extract lines around each `read_labels` call to verify the arguments being passed.

# Test: Extract context around `read_labels` calls.
rg 'read_labels' --glob '*.py' -C 3

Length of output: 11597

if not skeleton:
skeleton = JABS_DEFAULT_SKELETON
tracks = {}
Expand Down Expand Up @@ -166,7 +167,7 @@ def read_labels(
)
if new_instance:
instances.append(new_instance)
frame_label = LabeledFrame(Video(video_name), frame_idx, instances)
frame_label = LabeledFrame(video, frame_idx, instances)
frames.append(frame_label)
return Labels(frames)

Expand Down
10 changes: 8 additions & 2 deletions sleap_io/io/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,20 @@ def load_slp(filename: str) -> Labels:
return slp.read_labels(filename)


def save_slp(labels: Labels, filename: str):
def save_slp(
labels: Labels, filename: str, embed: str | list[tuple[Video, int]] | None = None
):
"""Save a SLEAP dataset to a `.slp` file.

Args:
labels: A SLEAP `Labels` object (see `load_slp`).
filename: Path to save labels to ending with `.slp`.
embed: One of `"user"`, `"suggestions"`, `"user+suggestions"`, `"source"` or
list of tuples of `(video, frame_idx)` specifying the frames to embed. If
`"source"` is specified, no images will be embedded and the source video
will be restored if available.
"""
return slp.write_labels(filename, labels)
return slp.write_labels(filename, labels, embed=embed)


def load_nwb(filename: str) -> Labels:
Expand Down
Loading
Loading