Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MacOS] Wrong executable architecture in package .._macosx_11_0_arm64 #52

Closed
marcelhuberfoo opened this issue Sep 17, 2023 · 9 comments
Closed

Comments

@marcelhuberfoo
Copy link

marcelhuberfoo commented Sep 17, 2023

I discovered that the distribution package clang_tidy-15.0.2.1-py2.py3-none-macosx_11_0_arm64.whl falsely contains the x86_64 binary.
I extracted the downloaded wheel using unzip and used the file utility to examine the architecture.
file clang_tidy-15.0.2.1-py2.py3-none-macosx_11_0_arm64/clang_tidy/data/bin/clang-tidy shows Mach-O 64-bit executable x86_64 instead of Mach-O 64-bit executable arm64.
As I am explicitly not wanting to use the rosetta emulator on my MacOS M1 machine, I am not able to use clang-tidy for now.
I downgraded to using 15.0.2 which seems to properly install the arm64 binary.

@marcelhuberfoo
Copy link
Author

The issue still appears in the latest clang_tidy-18.1.1-py2.py3-none-macosx_11_0_arm64.whl package where the packaged binary is still x86_64 instead of arm64 .
Unfortunately, on Apple Silicon machines, the rosetta2 frameworks kicks in silently and therefore you won't detect such an issue easily as it will just run the x86 binary on your arm64 without letting you know.

@renefritze
Copy link
Collaborator

Trying to address this in #74
Would be great if you could check the produced wheel from the manual build I triggered @marcelhuberfoo. Should be ready in a few hours.

@marcelhuberfoo
Copy link
Author

Trying to address this in #74 Would be great if you could check the produced wheel from the manual build I triggered @marcelhuberfoo. Should be ready in a few hours.

You're potentially forcing me to upgrade to Sonoma (macos-14) now 🥲, up to now, I could still use Ventura (macos-13) without too much of a hassle 😄
Nevertheless, I learned how to make use of restricted dependencies.

@marcelhuberfoo
Copy link
Author

marcelhuberfoo commented Jul 1, 2024

Unfortunately, the build was still targeting x86_64 when looking to the logs as well as according to the wheel check which actually failed the build.
I played around a bit to find the proper options required to cross build, and managed to use the following cmake configuration command to successfully build an arm64/AArch64 binary:

cmake -S llvm -B XBLD -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" \
-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=11.0 -DCMAKE_OSX_ARCHITECTURES:STRING=arm64 \
-DCMAKE_BUILD_TYPE:STRING=Release -DCMAKE_SYSTEM_NAME=Darwin \
-DCMAKE_INSTALL_PREFIX=$HOME/xbld -DLLVM_HOST_TRIPLE=arm64-apple-darwin22.6.0 \
-DLLVM_TARGETS_TO_BUILD=AArch64 -G Ninja -DHAVE_POSIX_REGEX=0 \
-DHAVE_STEADY_CLOCK=0 -DLLVM_ENABLE_ZSTD:STRING=OFF

As we are crosscompiling, the check-llvm test target cannot be executed and unfortunately I didn't find the correct options to suppress building the tests.

ninja -C XBLD install

A set of options was copied from the manual build job you referenced with additional options to direct cmake to actually cross compile.

  • -DCMAKE_SYSTEM_NAME=Darwin
  • -DLLVM_HOST_TRIPLE=arm64-apple-darwin22.6.0 # $(gcc -dumpmachine)
  • -DLLVM_TARGETS_TO_BUILD=AArch64 # probably the most important one
  • -DHAVE_POSIX_REGEX=0 # in order to suppress build warnings
  • -DHAVE_STEADY_CLOCK=0 # in order to suppress build waringns
  • -DLLVM_ENABLE_ZSTD:STRING=OFF # to not fail linking due to incorrect libzstd.dylib architecture

I hope this helps you figuring out the correct options to set for the build.
Thank you for your support.

@renefritze
Copy link
Collaborator

Thanks for that @marcelhuberfoo !
I'll ping you when I got an actual artifact to try.

@marcelhuberfoo
Copy link
Author

I played around a bit more and even found a configuration where you could reduce to one build job, producing a so called universal binary containing code for both architectures in the same file. This file is actually more or less double the size but at least executable out of the box on both architectures.
For this, you would need to change the following flags:

  • -DCMAKE_OSX_ARCHITECTURES:STRING="arm64;x86_64"
  • -DLLVM_TARGETS_TO_BUILD="AArch64;X86"

and you could leave out LLVM_HOST_TRIPLE as it is even not clear what it should be set to.

@renefritze
Copy link
Collaborator

Might actually be much simpler in that we've hit this
The pre-installed 3.8 interpreter isn't actually native itself!
New build with the workaround in place is now running https://github.com/ssciwr/clang-tidy-wheel/actions/runs/9841436874/job/27168025877

@marcelhuberfoo
Copy link
Author

Might actually be much simpler in that we've hit this The pre-installed 3.8 interpreter isn't actually native itself! New build with the workaround in place is now running https://github.com/ssciwr/clang-tidy-wheel/actions/runs/9841436874/job/27168025877

Thanks @renefritze, a proper arm64 binary was provided by the generated wheel 🎉
Seems we are not far away from having a proper build soon 😃

@renefritze renefritze mentioned this issue Jul 9, 2024
@renefritze
Copy link
Collaborator

New wheels for tidy 18.1.8 should hit PyPi in a few hours

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants