Consider releasing sleap-io on conda-forge

[niksirbi]

Hi!

We are considering releasing movement on conda-forge, to allow for a one-line installation of the package (including all current, or future non-pip dependencies).

The recommended way is for all our dependencies to be on conda-forge as well. Currently sleap-io is our only dependency which is not there.

I was wondering if you are planning to do a conda-forge release at some point. Apart from the specific needs of movement, this will make it easier for more packages to depend on sleap-io.

I completely understand that you may not have the bandwidth to do that currently, so as an option, I'm offering to help with getting sleap-io on conda-forge. I've done some initial investigations, and I think it shouldn't be too hard (famous last words 😉). Out of sleap-io's current dependencies, only ndx-pose seems to be missing from conda-forge, so it will likely come down to convincing them as well.

Anyhow, if you are willing to consider conda-forge in principle, let me know, and I can scope out the necessary work and help with most of it (incl. raising the issue with ndx-pose developers).

[talmo]

Hi @niksirbi,

Thanks for the suggestion! We've been wanting to do this as well and would certainly appreciate the help since our usual experiences with conda packaging have been pretty painful (but worth it).

If the blocker is ndx-pose, that seems very doable. I'll post on their repo and link back to this. We have a hackathon coming up with the NWB folks in late February which would be a great time to get this done if it's not already by then.

[niksirbi]

Thanks for the positive response @talmo !

It would be great for the whole ecosystem to go on conda, enabling people to use the packages together.
It requires some work to make the first release, but subsequent updates are simpler and mostly handled by conda-forge's bots.
Thanks for starting the discussion on ndx-pose. That has minimal dependencies, so getting it on conda-forge should be straightforward. I'm not attending the Pre-CoSyNe hackathon, but since I'm in the same timezone, would be open to participating remotely, should my help be needed.

Regarding sleap-io getting released on conda, I've so far identified 3 potential issues:

• ndx-pose not being on conda-forge, which we already discussed
• av on conda-forge not being completely up-to-date with the pip releases (I've already inquired about this with the av maintainers).
• the PyPI release of sleap-io not including the sdist. This should be easily fixable by modifying the build workflow. For reference, here's the list of actions/workflows we use for the same purpose. If the sdist is on PyPI, we could use grayskull to easily initialise the meta.yaml file for conda-forge.

There may be other things to be uncovered down the line, but I'm hopeful that the above may be the only 3 hurdles.

[niksirbi]

ndx-pose is now available on conda-forge 🎉 , so one of the blockers has been removed.

[niksirbi]

I was wondering if there has been any progress on this front. Don't hesitate to ping me if help is needed.

[talmo]

Hey @niksirbi! Sorry for the delay -- been traveling and just catching up now.

I did add sdist building to our workflow in #74, so v0.0.13 on PyPI is now compatible with grayskull.

That was nice to start off the conda build file, but it turns out not to be too complicated of a conda build anyway.

The next step I hadn't looked into was getting it uploaded onto conda-forge. I didn't see anything about conda-forge uploading in your workflows repo or in the movement one?

Do you handle that separately? Are you just following these exact steps from their docs? How do you do the automation?

[niksirbi]

Hey @talmo, thanks for catching up on this.

Releasing on conda-forge is different to PyPI, and we don't do it via our usual CI workflows.
We basically follow the steps outlined in the conda-forge docs you've linked.

It's some work the first time you do it for a package, because one has to go through the various steps and official checks.
But for subsequent releases the process should be quite smooth.

The way I understand it is that once the conda-forge/{package}-feedstock repository is created, new PyPI releases will automatically trigger a bot to open a PR on that feedstock repo. In most cases where the dependencies haven't changed, you (and others marked a maintainers of that feedstock) can simply accept and merge that PR to build a new conda-forge release. If a new release includes changes in dependencies, you might need to modify a few lines in the PR opened by the bot. @adamltyson can correct me if I'm wrong on this.

In case it helps, here are some examples for another package of ours that we recently published on conda-forge:

• the staged-recipe PR
• the corresponding feedstock repo

[talmo]

WIP: conda-forge/staged-recipes#26036

[niksirbi]

Hi @talmo,

We've done some investigations on the segfault crushing the conda recipe build, and it comes down to OpenCV, though we couldn't pinpoint the problem exactly.

Do you see any way around that problem? For us, releasing movement on conda-forge is a priority, as this is how we also release and distribute all of our other tools, and we'd like them all to be easily installable and interoperable. The sleap-io dependency is the only thing blocking us right now.

[talmo]

Now done in conda-forge/staged-recipes#26036!

Feedstock for reference: https://github.com/conda-forge/sleap-io-feedstock

Anaconda page: https://anaconda.org/conda-forge/sleap-io