Heads up: tox 4.0.13 breaks our tox.ini

[dataflake]

See tox-dev/tox#2747

@icemac @mauritsvanrees A change in tox 4.0.13 broke the expression we use to distinguish between environments when running the buildout. This is now breaking tons of automated tox runs on GitHub Actions.

[testenv]
...
commands_pre =
    py27,py35: {envbindir}/buildout -nc {toxinidir}/buildout4.cfg buildout:directory={envdir} buildout:develop={toxinidir} install test
    !py27,!py35: {envbindir}/buildout -nc {toxinidir}/buildout.cfg buildout:directory={envdir} buildout:develop={toxinidir} install test
...

The negation doesn't work anymore. py27 and py35 suddenly match both commands_pre invocations so both are run. I cannot find any place in the official documentation where negated expressions are explained, so I don't even know if what we had been doing was already deprecated and should have been rewritten.

I am hoping they will either point me to a fixed syntax or fix their parsing.

[gaborbernat]

This change is not just 4.0.13; it's any 4.0 version; the reporting claims otherwise but I get the same behavior with 4.0. I'm surprised this worked in tox 3 because we use the , as an OR operator... So py27,py35 means the environment is py27 or py35, so we match. Then !py27,!py35 means not py27 or not py35, so both lines are true. What would you expect the second conditional to explode to?

[dataflake]

The specific expression we used, !py27-!py35, worked with tox 4.0.12 and broke with 4.0.13. I tested this manually today. The change in behavior broke automated testing for about 350 packages in the zopefoundation GitHub organization, so I would not classify it as a "minor" bug.

Can you suggest an expression hat works with tox 4.0.13 and up that means "environment is NOT py27 AND NOT py35"? There's nothing I can find in the official documentation.

[gaborbernat]

That's probably because there's no official support for that in tox 4. PR to add it is welcome.

[dataflake]

Looks like we have relied on a syntax that was not documented and not supported.

I do not plan on becoming a tox developer, I already have full or part responsibility for way too many projects.

I'll experiment a bit more tomorrow to see if I can fix the issue in the tox configuration. If all else fails we can change the configuration templates to create separate environments for py27 and py35

[dataflake]

This issue is most likely superceded by #177. I am sure we will have a consensus about cutting all support for Python 2.7 and 3.5 at this point.

[dataflake]

By the way, tox 4.0.15 may contain a fix. I haven't tested it yet.

[dataflake]

Manual testing proves that the current tox version (4.0.16 right now) fixes the negated expression issue. Specifically, the expression we have used in many places, !py27-!py35, works again.

I'll close this issue, the underlying problem is fixed and we're already working on a different solution.