check-manifest doesn't understand cases where VCS contents legitimately differ from sdist

[mgedmin]

Sometimes you want to put generated files in the sdists, but you don't want to have them under version control. E.g. when you use Cython, you want to generate .c files and include them in sdists so your users don't have to have Cython installed to build your software -- all they'd need is a C compiler.

Currently check-manifest is ignorant of situations like these and warns about those generated files missing from VCS.

I think this is most likely a documentation bug:

• figure out what's the right way to use check-manifest with a Cython-using package (e.g. put ignore = *.c in a setup.cfg?)
• document it in the appropriate place
• (maybe) have check-manifest suggest the ignore = *.c if it sees *.c or cython.dat in the "missing from VCS list
• (maybe) include cython.dat in the default ignore list?

Some research is needed, e.g. what even is cython.dat and should it be present in sdists?

[tysonclugg]

-1

The point of an sdist is not to install stuff -- although you can because being able to build or install a package is a side-effect of having all the sources available. The point of an sdist is to distribute the source.

You should use wheel to produce installable distributions, since then you don't even need a C compiler to install them.

[vegarsti]

I also find this confusing. Is it possible to ignore this warning? At the moment, I'm not able to use it in a Jenkins pipeline because it returns the wrong exit code. But maybe I'm misunderstanding how to use it, that it's not meant for this use? How did you solve this @mgedmin?

[mgedmin]

I don't have packages that distribute generated files ;)

Which is why this issue is staying open so long -- I don't have a test case, and other people are for some reason not opening pull requests.

@vegarsti do you have an open source githup repo you could point me at, that suffers from this problem?

[vegarsti]

@mgedmin Unfortunately not yet, need to tick a few boxes first. Will let you know! Thanks for being helpful!

[nixphix]

@mgedmin we have this situation in sklearn I'm trying to ignore some files that are in sdist but not in VCS check this github action log

And ignoring in setup.cfg is not ignoring the missing files in VCS

[mgedmin]

nixphix/scikit-learn@bd3b368#diff-380c6a8ebbbce17d55d50ef17d3cf906R32 is not using the right syntax

[tool.check-manifest] is for pyproject.toml files; setup.cfg needs a simple [check-manifest] section.

Let me know if that resolves your problem!