List of non-python 3 packages

[ocefpaf]

• ulmo
• suds (suds-jurko)
• pyke
• unit_conversion
• numdifftools (fixed in Updated numdifftools 0.9.13 #574)
• pyproj
• pyoos
• pycsw (WIP)
• iris (WIP)
• utilities (iris)
• tardis (iris)
• pywafo
• octant
• pastescript (will not fix)
• pygdp (WIP)
• optcomplete (will not fix)
• petulant-bear (will not fix)
• pyhum
• pydap (will not fix)
• pyfvcom (will not fix)
• compliance-checker
• cc-plugin-glider (compliance-checker)
• pyseidon
• wicken
• mocsy (compiler issue no mocsy on py3k #567)

[kwilcox]

It's super impressive that this list is so short! Awesome stuff @ocefpaf

[ocefpaf]

I might be missing something, but those are known failure. If you ever hit a wall let me know,

PS: pyke has a Python3 version, but we are having issues with it now. Also, suds has a substitute for Python3 (suds-jurko), but I am planning to deprecated that package. I guess we no longer use it.

[tomkralidis]

We have an an open issue in pycsw for Python 3 support. @QuLogic has done excellent work in helping bump OWSLib and geolinks to Python 3 support, but I'm not sure about the rest of pycsw dependencies (SQLAlchemy, pyproj). Any pycsw Python 3 help would be very much appreciated.

[ocefpaf]

I pyproj is already Python3 (and available in the IOOS channel 😉 )

I did not test SQLAlchemy, but the classifiers at PyPI say Python3 is available.

I am not sure the amount of work that is needed in pycsw, but I will add to my list for a long rainy weekend.

[QuLogic]

For some reason pyproj is listed as a blocker; maybe they need to update their classifiers on PyPI. OWSLib is a blocker only because it's limited to <0.9.0.

[tomkralidis]

Looks like @ocefpaf is working on it and getting there: https://github.com/ocefpaf/pycsw/tree/py3k

[ocefpaf]

It is raining here...

[rsignell-usgs]

I am contributing by not harrassing @ocefpaf this afternoon. 😸

[ocefpaf]

I am contributing by not harrassing @ocefpaf this afternoon. 😸

😜

[emiliom]

That's great, @ocefpaf ! Regarding suds:

Also, suds has a substitute for Python3 (suds-jurko), but I am planning to deprecated that package. I guess we no longer use it.

suds is an ulmo dependency, for SOAP access, for CUAHSI WaterOneFlow/WaterML 1 access. If the intent is to keep ulmo on the ioos channel, suds would have to stay. I'm aware that suds has been abandoned, and I've seen suds-jurko but never tried it; don't know how much effort it'd be to replace suds with suds-jurko in ulmo. But @dharhas is currently overhauling ulmo, so this wouldn't be a time to invest in the current codebase. After that refactoring, my understanding is that the suds dependency would likely become an optional plugin.

Alternatively, @dharhas also has an ulmo conda recipe and binstar channel, but I don't know how actively maintained it is, or whether relying on an outside channel may create inconsistencies with stuff from the ioos channel. FYI, I use ulmo regularly.

[ocefpaf]

If the intent is to keep ulmo on the ioos channel, suds would have to stay.

Our deprecation process remove unsupported recipes from the repository but the binaries do stay in the channel. Since suds is unsupported it makes sense to deprecate it. And all the packages that depends on suds will have to do with the last available version.

The reason for deprecation is to lessen the burden on our numerous maintainers 😉

PS: I am watching @dharhas' ulmo refactoring close and I will keep our recipe up-to-date.

[emiliom]

Thanks for the explanation. Just goes to show my ignorance on binstar channel & recipe repo intricacies.

Glad you're watching the ulmo refactoring! I'm starting to suspect that days are longer than 24 hours out there in the Nordeste :) Or you have a clone.

[ocefpaf]

than 24 hours out there in the Nordeste :) Or you have a clone.

Nope. Just lots of rain due to a strong ENSO. We are not getting many chances to go to the beach this year. Well.. At least we will get more coding done 😉

[dharhas]

@ocefpaf @emiliom

so for kicks and giggles a few days ago, I ran python futurize on the current version of ulmo and tried running the tests on python 3. About 70% of the tests passed and quite a few of the rest seemed to be a unicode vs string issue. So that gives me more confidence about making ulmo python 3 compatible. I'm hoping that suds-jurko is a drop in replacement for suds. If it isn't I'm not sure what alternatives we have.

If possible I'd like to make the IOOS channel the primary place to get ulmo and drop my binstar/anaconda channel. I don't really maintain the ulmo channel very well and don't have an automated build system in place.

In another note, I'm considering keeping ulmo as a single package rather than splitting it up into optional packages. Basically multiple packages makes for more overhead in terms on managing releases/running tests etc and since I'm pretty much the only one maintaining ulmo I'm not sure it makes sense to split it up.

Bottom line. In the near (for some value of near) future ulmo should be py3 compatible. I need it to be py3 compatible for another project.

• dharhas

[ocefpaf]

About 70% of the tests passed and quite a few of the rest seemed to be a unicode vs string issue.

Awesome!

If possible I'd like to make the IOOS channel the primary place to get ulmo.

It definitely is. Just ping us when you have a newer version of ulmo and we will build it 😉

[emiliom]

Re: ulmo -- Great on all counts ("consolidating" to IOOS channel, py3 future, etc).

Great also to see @ocefpaf has just added a recipe for suds-jurko! Like @dharhas said, hopefully it'll be a drop-in replacement for suds. Dharhas, I can volunteer to help test it, sometime after this week.

[ocefpaf]

Pinging @lukecampbell ❗
(See the last item on the list.)

[rsignell-usgs]

But it is kind of amusing that the compliance-checker is not Python 3-compliant, no? 😝

[lukecampbell]

I'm knee deep in some major changes to compliance checker, I'll see if I can't bring it up to compliance soon.

[dharhas]

@emiliom @ocefpaf @rsignell-usgs

So after much unicode/string/bytes pain (I still don't think I completely get how all of that works), I have a py2.7/3.4 compatible version of ulmo that is passing tests on travis. Planning to merge it in as ulmo 0.8, checkout ulmo-dev/ulmo#112

if any of you are using ulmo in scripts please try them with py2/3. Tests are passing but I don't have complete coverage so ymmv.

[ocefpaf]

So after much unicode/string/bytes pain

So you played everybody favorite game encode/decode whack-a-mole 😉

(I still don't think I completely get how all of that works)

Me neither. I just hope to do this kind of thing only once.

Planning to merge it in as ulmo 0.8

As soon as you merge I will update the package here. Thanks @dharhas!

[dharhas]

yup. plus some BytesIO/StringIO whack-a-mole for good measure.

[dharhas]

@ocefpaf

ulmo 0.8.0 is tagged v0.8.0 on github and uploaded to pypi. I have CI setup with AppVeyor and Travis and everything looks good now. Look at the py2/3_conda_environment.yml files to see the packages ulmo needs. There is a hard crash for pytables > 3.1.1 on windows/py3.4

[ocefpaf]

Almost there. We need ulmo-dev/ulmo#113.

Quick questions:

1. Is mock a dependency or a test dependency? It is not listed in the tests_require and the comment on setup.py makes me think it is a required dependency.

2. lxml is listed in setup.py, but absent in the py2/3_conda_environment.yml. I guess it is still a dependency, right?

[dharhas]

Ok I've merged the pull request. Before I tag a 0.8.1 release I thought I'd check a few things.

1. After checking, mock is used in two places. a) For tests and b) To enable readthedocs to build the docs without having to install pytables (pytables is an optional dependency). Without pytables the usgs.nwis.hdf5 stuff doesn't work and all hdf5 related tests fail.

2. lxml is not used directly by ulmo but is used by beautifulsoup4. When you conda install bs4 you get lxml automatically but I think in general it is a optional dependency of bs4 so setup.py explicitly lists it.

So should I explicitly add lxml to the conda_environment files?

[ocefpaf]

Without pytables the usgs.nwis.hdf5 stuff doesn't work and all hdf5 related tests fail.

OK. I will add it as dependency.

So should I explicitly add lxml to the conda_environment files?

I think that it is best. You never know if you will not lxml or not when installing bs4. And I see that ulmo imports lxml in a few places (here and here).

[dharhas]

Ok 0.8.1 is on pypi now and ulmo is pip installable (and I worked out how to use the pypitest server before making the actual release, so I don't use up version numbers on broken releases)

[ocefpaf]

PR #434 is using PyPI. Thanks @dharhas!

[QuLogic]

For Python 3.3+, you could just import unittest.mock and leave out the mock dependency. The package is just a backport, and up until the release of 3.5, probably wouldn't be much difference.

[ocefpaf]

Thanks @QuLogic! I will look into that.

[ocefpaf]

This list is outdated and most of these packages either already have Python 3 support or is abandoned and never will.