Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build for linux-ppc64le #44

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

agriyakhetarpal
Copy link
Member

@agriyakhetarpal agriyakhetarpal commented Oct 13, 2024

Checklist

  • Used a personal fork of the feedstock to propose changes
  • Bumped the build number (if the version is unchanged)
  • Reset the build number to 0 (if the version changed)
  • Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
  • Ensured the license file is being packaged.

@agriyakhetarpal agriyakhetarpal changed the title Patch 1 Build for linux-ppc64le Oct 13, 2024
@agriyakhetarpal
Copy link
Member Author

@conda-forge-admin, please rerender

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@agriyakhetarpal
Copy link
Member Author

agriyakhetarpal commented Oct 13, 2024

I don't have full access to a computer right now, so this PR has been created from GitHub's web interface. I've triggered a rerender of the feedstock in the meantime and I can dive into any errors that arise with the compilation (I expect a few) in a while.

Edit: I've re-rendered locally with edd91c3.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the GitHub actions workflow below for more details. You can also ping conda-forge/core for further assistance or you can try rerendering locally.

The following suggestions might help debug any issues:

  • Is the recipe/{meta.yaml,recipe.yaml} file valid?
  • If there is a recipe/conda-build-config.yaml file in the feedstock make sure that it is compatible with the current global pinnnings.
  • Is the fork used for this PR on an organization or user GitHub account? Automated rerendering via the webservices admin bot only works for user GitHub accounts.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/11314779755.

@agriyakhetarpal
Copy link
Member Author

sed: can't read /home/conda/feedstock_root/build_artifacts/zig_1728828681553/_build_env/ppc64le-conda-linux-gnu/sysroot/usr/lib64/libm.so: No such file or directory should be fixed if I set SYSROOT_ARCH as powerpc64le instead of ppc64le.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.

Here's what I've got...

For recipe/meta.yaml:

  • Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [50, 54]

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@agriyakhetarpal
Copy link
Member Author

agriyakhetarpal commented Oct 13, 2024

I see that y'all have tried this already 😁 cross-linking to ziglang/zig#20335, and to #25 (comment) for me to refer further. ziglang/zig-bootstrap#113 (comment) mentions that we need ziglang/zig#17749.

ziglang/zig#21650 and ziglang/zig#21651 discuss further support for this target, and essentially the blocker here is ziglang/zig#5927. I don't think we should support ELF v1 since Zig will support just v2 in the next release – ziglang/zig#21310 was just merged. So, we'll need some of those other issues to move further – I can rebase this PR then.

In the meantime, I'll continue to rely on https://github.com/ziglang/zig-pypi/ which can repackage the official ppc64le builds for me as wheels :)

Side note: has the usage of those wheels to build a no-arch conda package here been discussed before? It would need a patch to expose the zig entry point in the system's PATH, but that should all work. never mind, that would add a dependence on Python, which, strictly speaking, the "language" does not need :P

@MementoRC
Copy link
Contributor

I did not get notified on this effort. The issue I ended-up shocking on is that LLVM 18 does not support some of the PPC64 relocations that the compilation generates. I have tried 3/4 methods to build it, without success so far

@jakirkham
Copy link
Member

Perhaps this observation about removing the -fno-plt compiler flag is helpful

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@jakirkham
Copy link
Member

Perhaps this is related to the new CI error, which also notes relocation issues

ziglang/zig#9478

@MementoRC
Copy link
Contributor

MementoRC commented Oct 28, 2024

Well, Andrew took a brief look at the issue I had posted and seemed to acknowledge that LLVM for ppc64le was missing these relocation, but I admit that I don't remember the details. I tried so many different ways of building it and I think the most promising was a native build on Travis, that's where I saw the relocation issues after patching a bunch of stuff in zig asm registers - Ideally, I would take another in-depth look at it when the qemu-user-execve is approved - You know, the one that I redirected you toward ... Currently, I have a very hard time using the cmake-based cross build that works fine for aarch64 (I prefer to use cmake rather than cross-zig)

@MementoRC
Copy link
Contributor

Same issue with zig 0.14.0 and LLVM 19

@alexrp
Copy link

alexrp commented Nov 18, 2024

ziglang/zig-bootstrap#113 (comment) mentions that we need ziglang/zig#17749.

Just to clarify, that issue is specific to powerpc64. For powerpc64, glibc requires ELFv1, while musl requires ELFv2. This means that Zig can't currently produce working binaries for powerpc64-linux-gnu, but binaries for powerpc64-linux-musl (and powerpc64-linux-none) are fine.

powerpc64le is a much newer port, for which both glibc and musl require ELFv2. Both LLVM and Zig have full support for ELFv2.

@agriyakhetarpal
Copy link
Member Author

Thanks for the context! I guess we'll have to wait for things to move upstream...

@MementoRC
Copy link
Contributor

MementoRC commented Nov 18, 2024

powerpc64le is a much newer port, for which both glibc and musl require ELFv2. Both LLVM and Zig have full support for ELFv2.

Are you implying that the issue we are seeing with ppc64le on those PRs is due to a misundertanding on our part? and that ppc64le build is expected to work? We need to build with our existing conda LLVM shared library, which seem to be missing relocations that the zig build is emitting (though, I admit my understanding of the details is poor)

@alexrp
Copy link

alexrp commented Nov 18, 2024

Are you implying that the issue we are seeing with ppc64le on those PRs is due to a misundertanding on our part?

No, I just wanted to clarify that the linked zig-bootstrap issue is specific to powerpc64 and so doesn't apply here since this PR is for powerpc64le.

and that ppc64le build is expected to work?

We are able to bootstrap a Zig compiler for powerpc64le-linux-(gnu,musl) using zig-bootstrap. But I haven't tried building the compiler on an actual powerpc64le system.

We need to build with our existing conda LLVM shared library, which seem to be missing relocations that the zig build is emitting (though, I admit my understanding of the details is poor)

What seems to be happening is that the built LLVM artifacts (and libc, etc) contain relocations that LLVM/LLD/Clang just don't support/use yet, but which GCC/binutils do. So when you try to link the Zig compiler against those artifacts using LLD, LLD doesn't know what to do with them.

The proper solution is to teach LLD how to process these relocations.

A more immediate workaround would be to avoid building any LLVM artifacts with GCC, and instead use Clang, since Clang won't emit these relocations.

This still leaves the libc relocations, though, but perhaps you can side-step that issue by explicitly compiling for powerpc64-linux-gnu. This will make Zig link against a stub libc.so that's link-compatible with glibc, but is built with LLVM's assembler, so it won't contain any of those problematic relocations.

@MementoRC
Copy link
Contributor

@alexrp Thank you for clearly enunciating the problem that I very superficially understood, as well as, providing a path to a solution (which I very superficially understand ... lol). I will get on it and may try to ping you when I have made some progress. Thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants