Replace sage.libs.giac with new optional package sagemath-giac

[orlitzky]

As the last step in #38668, we move sage/libs/giac into a separate package and make it optional. This module is already mostly independent of sage -- there is nothing in sagelib that depends on it unconditionally. Its main use is as an integration backend, where it is technically optional because we try whatever backends are available in succession and expect some to fail.

In fact, the modules themselves are already optional if you are using meson. The sage/libs/giac directory is simply skipped when meson cannot find giac. It is not possible to do the same thing using the build system for sage-the-distro, however, which motivates the separate package. Many reasons are given in #38668 for why you might want to avoid giac, but the main reason for me right now is because it's not portable: it doesn't build and/or run on systems where the rest of sage works fine, including the one I am using most of the time.

As a prerequisite, I have replaced the old deprecated sage-cython script with a new, much shorter one. The new one simply ensures that we are invoking cython using the python that will be used to run sage. If we invoke it with (say) the system python instead, the search path for *.pxd files will be wrong. (Naturally, this breaks the build process for the new sagemath-giac package).

The new package uses the namespace sagemath_giac to work around missing namespace support in cython, but sage.libs.giac re-exports everything under the old name for backwards compatibility.

Finally, this is a draft because the package info is pointing at my own personal GH repo, but that will change soon.

[github-actions]

Documentation preview for this PR (built with commit 454c025; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[antonio-rojas]

In fact, the modules themselves are already optional if you are using meson. The sage/libs/giac directory is simply skipped when meson cannot find giac. It is not possible to do the same thing using the build system for sage-the-distro, however, which motivates the separate package.

How is giac integration different from bliss/meataxe/sirocco that makes it impossible for it to be optional while still living in the main sage source tree?

[orlitzky]

How is giac integration different from bliss/meataxe/sirocco that makes it impossible for it to be optional while still living in the main sage source tree?

It's not. Matthias has a sagemath_giac in his fork of Sage along those same lines, but however you go about it, it's a good amount of work. (You can't just add a --disable-giac flag that skips sage/libs/giac.)

What you'll find at https://github.com/orlitzky/sagemath-giac is a normal cython package. You can clone it, build it, test it, package it, etc. independent of sage-the-distro, using what you already know as a python developer. There's no magic, and even the doctests work with a simple python -m doctest or pytest. Considering that both approaches involve a significant amount of work, I find this to be a much friendlier design.

[antonio-rojas]

What you'll find at https://github.com/orlitzky/sagemath-giac is a normal cython package. You can clone it, build it, test it, package it, etc. independent of sage-the-distro, using what you already know as a python developer. There's no magic, and even the doctests work with a simple python -m doctest or pytest. Considering that both approaches involve a significant amount of work, I find this to be a much friendlier design.

I disagree - thinking outside the sage-the-distro framework I see a few disadvantages of this approach:

• This makes sagemath-giac a de-facto third-party package. When there are breaking changes in sagelib, you need to port stuff in two different places instead of having all code centralized in a single repo (adds burden to developers), and make sure everything is released in sync (adds burden to upstream maintainers) - we're seeing this now with the pari 2.17 upgrade and SnapPy.
• You can no longer build and package all components (standard and optional) from a single repo - you have to either combine several sources at build time or create separate packages (adds burden to downstream packagers).

Since, as you said, this works properly in meson and we're moving towards making meson the default buildsystem, won't this eventually fix itself?

[orlitzky]

• This makes sagemath-giac a de-facto third-party package. When there are breaking changes in sagelib, you need to port stuff in two different places instead of having all code centralized in a single repo (adds burden to developers), and make sure everything is released in sync (adds burden to upstream maintainers) - we're seeing this now with the pari 2.17 upgrade and SnapPy.
• You can no longer build and package all components (standard and optional) from a single repo - you have to either combine several sources at build time or create separate packages (adds burden to downstream packagers).

Fair concerns, but in this case I would not be too worried:

• These modules are very lightly integrated with sage and should not break when sage upgrades. If you look through the git log, you'll see a few module names changing in import statements, but nothing else. (And we should have a year to sync those while the old names are deprecated.)
• On the contrary, because it is largely independent of sage, it will be less work to package it once than it would be to package (the same exact code) multiple times with each release of sage, like we do with sagemath_bliss and friends. The most common changes are updates to the doctest output due to changes in giac, which become much easier to fix and release in a minor bump to sagemath-giac without having to wait 6 months for the next version of sage while your tests fail.
• It's becoming an optional package, so it requires no extra work. You can do nothing and upgrade never :)

Since, as you said, this works properly in meson and we're moving towards making meson the default buildsystem, won't this eventually fix itself?

Absolutely, this was a tempting approach. But giac is completely broken on both types of systems I have. It doesn't build on anything other than x86_64/arm64, and even on x86_64, it crashes under a hardened glibcxx. How long is eventually?

[orlitzky]

It is maybe worth mentioning that, once upon a time, this was already a separate package. It was bundled with sagelib when it was made a standard package to avoid a circular dependency sagelib -> sagemath_giac -> sagelib. But today, there is no sagelib -> sagemath_giac dependency. All of the integration backends are drop-ins and can fail without issue.