Document cross-compilation on macOS #128
Conversation
tamird
force-pushed
the
cross-compile-mac
branch
from
7153b26 to
85ffeba
Compare
README.md
Outdated
| ```bash | ||
| AYA_BUILD_EBPF=true cargo build --package {{project-name}} \ | ||
| --release --target=${ARCH}-unknown-linux-musl \ | ||
| --config 'target.${ARCH}-unknown-linux-musl.linker = "${ARCH}-linux-musl-ld"' |
There was a problem hiding this comment.
Don't you also need to set CC=x86_64-linux-musl-gcc?
also if you're passing --target - where you hardcode ${ARCH} - then it's easier to do -C linker=${ARCH}-linux-musl-ld ?
There was a problem hiding this comment.
Would you mind testing it? The docs in both https://github.com/FiloSottile/homebrew-musl-cross and https://aya-rs.dev/book/aya/crosscompile/ don't mention setting CC.
I was trying to avoid setting environment variables (and to pass -C linker=... you need to pass it in RUSTFLAGS).
There was a problem hiding this comment.
Would you mind testing it?
I cross compile every day from aarch64 (linux vm running on my mac) to x86, this is my cmdline
PATH=$HOME/src/musl-cross-make/output/bin:$PATH CC=x86_64-linux-musl-gcc RUSTFLAGS="-C linker=x86_64-linux-musl-gcc" cargo build --target=x86_64-unknown-linux-musl --release
musl-cross-make is a musl-cross distro for linux, similar to filosottile's for mac
If we don't set CC, the sys libraries (libopenssl etc) get compiled with the host compiler, not the cross compiler, then things fail at link time since you have two different ELF machine targets
and to pass -C linker=... you need to pass it in RUSTFLAGS
Fair, although I'm not sure why? The rlibs (deps) shouldn't be linked at all, only the final main crate gets linked, so it should work 🤔
There was a problem hiding this comment.
Setting CC isn't necessarily going to make C libraries build correctly; it's up to those libraries' build scripts to inspect that environment variable. It isn't part of the contract between the user, cargo, and rustc. Including that in our docs amounts to documenting a convention which may or may not be respected by various crates in the ecosystem.
I'm beginning to rethink whether we should document this stuff at all. There's no longer anything special about cross-compiling programs generated by aya-template (apart from having to set AYA_BUILD_EBPF=true).
Thoughts?
There was a problem hiding this comment.
Setting CC isn't necessarily going to make C libraries build correctly; it's up to those libraries' build scripts to inspect that environment variable. It isn't part of the contract between the user, cargo, and rustc
it is, the cc crate which is used to compile virtually all -sys libraries honors CC etc. Besides, CC, CXX etc have been standard for decades.
There was a problem hiding this comment.
Yeah, but you're making my point for me: we'd be documenting those things rather than anything specific to aya-template.
There was a problem hiding this comment.
I don't think that we need to teach people 50 years of UNIX when we can just give them a command that works in 99% of the cases?
There was a problem hiding this comment.
To be clear I don't care if we have it in aya-template or in the book, as long as we have it somewhere, because we didn't use to and people used to ask
The instructions in the book are now out of date for projects generated with this template. Since the instructions are much shorter these days, put them in the README.
tamird
force-pushed
the
cross-compile-mac
branch
from
85ffeba to
4e89e57
Compare
|
not sure if I missed anything, but this doesn't seem to work: lielfridman@Liels-MacBook-Pro ebpf-auditer % AYA_BUILD_EBPF=true \
CC=${ARCH}-linux-musl-gcc \
RUSTFLAGS="-C linker=${ARCH}-linux-musl-gcc" \
cargo build --package ebpf-auditer --release --target=${ARCH}-unknown-linux-musl
Compiling ebpf-auditer v0.1.0 (/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer)
The following warnings were emitted during compilation:
warning: [email protected]: Compiling ebpf-auditer-ebpf v0.1.0 (/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer-ebpf)
warning: [email protected]: error: linking with `aarch64-linux-musl-gcc` failed: exit status: 1
warning: [email protected]: |
warning: [email protected]: = note: LC_ALL="C" PATH="/Users/lielfridman/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/opt/homebrew/opt/openjdk@21/bin:/opt/homebrew/opt/llvm/bin:/Users/lielfridman/flutter/bin:/Users/lielfridman/.local/bin:/Users/lielfridman/.rbenv/shims:/opt/homebrew/opt/mysql-client/bin:/Users/lielfridman/.nvm/versions/node/v20.11.0/bin:/Users/lielfridman/.pyenv/shims:/opt/homebrew/opt/openjdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Library/TeX/texbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/lielfridman/.cargo/bin:/Users/lielfridman/Library/Application Support/JetBrains/Toolbox/scripts:/Users/lielfridman/go/bin" VSLANG="1033" "aarch64-linux-musl-gcc" "--export-symbols" "/var/folders/nq/j57y61v91bqfgbx4qpy0jb_40000gn/T/rustcF5nzZH/symbols" "/var/folders/nq/j57y61v91bqfgbx4qpy0jb_40000gn/T/rustcF5nzZH/symbols.o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/ebpf_auditer-b96c95dc76996da2.ebpf_auditer.badf010e8e002827-cgu.0.rcgu.o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_log_ebpf-c9131489510da28c.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_log_common-e45c9f8c3c14f18f.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libnum_enum-e1b5a306ee8f5ea5.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf-1d4bf08ce8ea8fd8.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf_bindings-8cd067c8051990e8.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf_cty-3cf779c1bb043e7e.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/librustc_std_workspace_core-050bc42c86a0fc8d.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libcore-72540679a78b0f72.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libcompiler_builtins-83738ce13aeccd0f.rlib" "--cpu" "generic" "-o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/ebpf_auditer-b96c95dc76996da2" "-O3" "--debug"
warning: [email protected]: = note: aarch64-linux-musl-gcc: error: generic: No such file or directory
warning: [email protected]: aarch64-linux-musl-gcc: error: unrecognized command line option '--export-symbols'
warning: [email protected]: aarch64-linux-musl-gcc: error: unrecognized command line option '--cpu'
warning: [email protected]:
warning: [email protected]:
warning: [email protected]:
warning: [email protected]: error: could not compile `ebpf-auditer-ebpf` (bin "ebpf-auditer") due to 1 previous error
error: failed to run custom build command for `ebpf-auditer v0.1.0 (/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer)`
Caused by:
process didn't exit successfully: `/Users/lielfridman/Projects/Rust/ebpf-auditer/target/release/build/ebpf-auditer-776137e4fecc6045/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=AYA_BUILD_EBPF
cargo:rerun-if-changed=/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer-ebpf
cargo:warning= Compiling ebpf-auditer-ebpf v0.1.0 (/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer-ebpf)
cargo:warning=error: linking with `aarch64-linux-musl-gcc` failed: exit status: 1
cargo:warning= |
cargo:warning= = note: LC_ALL="C" PATH="/Users/lielfridman/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/bin:/opt/homebrew/opt/openjdk@21/bin:/opt/homebrew/opt/llvm/bin:/Users/lielfridman/flutter/bin:/Users/lielfridman/.local/bin:/Users/lielfridman/.rbenv/shims:/opt/homebrew/opt/mysql-client/bin:/Users/lielfridman/.nvm/versions/node/v20.11.0/bin:/Users/lielfridman/.pyenv/shims:/opt/homebrew/opt/openjdk/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Library/TeX/texbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/Users/lielfridman/.cargo/bin:/Users/lielfridman/Library/Application Support/JetBrains/Toolbox/scripts:/Users/lielfridman/go/bin" VSLANG="1033" "aarch64-linux-musl-gcc" "--export-symbols" "/var/folders/nq/j57y61v91bqfgbx4qpy0jb_40000gn/T/rustcF5nzZH/symbols" "/var/folders/nq/j57y61v91bqfgbx4qpy0jb_40000gn/T/rustcF5nzZH/symbols.o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/ebpf_auditer-b96c95dc76996da2.ebpf_auditer.badf010e8e002827-cgu.0.rcgu.o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_log_ebpf-c9131489510da28c.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_log_common-e45c9f8c3c14f18f.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libnum_enum-e1b5a306ee8f5ea5.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf-1d4bf08ce8ea8fd8.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf_bindings-8cd067c8051990e8.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libaya_ebpf_cty-3cf779c1bb043e7e.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/librustc_std_workspace_core-050bc42c86a0fc8d.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libcore-72540679a78b0f72.rlib" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/libcompiler_builtins-83738ce13aeccd0f.rlib" "--cpu" "generic" "-o" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf/bpfel-unknown-none/release/deps/ebpf_auditer-b96c95dc76996da2" "-O3" "--debug"
cargo:warning= = note: aarch64-linux-musl-gcc: error: generic: No such file or directory
cargo:warning= aarch64-linux-musl-gcc: error: unrecognized command line option '--export-symbols'
cargo:warning= aarch64-linux-musl-gcc: error: unrecognized command line option '--cpu'
cargo:warning=
cargo:warning=
cargo:warning=
cargo:warning=error: could not compile `ebpf-auditer-ebpf` (bin "ebpf-auditer") due to 1 previous error
--- stderr
thread 'main' panicked at ebpf-auditer/build.rs:153:9:
assertion `left == right` failed: cd "/Users/lielfridman/Projects/Rust/ebpf-auditer/ebpf-auditer-ebpf" && env -u RUSTC -u RUSTUP_TOOLCHAIN CARGO_CFG_BPF_TARGET_ARCH="aarch64" "cargo" "build" "-Z" "build-std=core" "--bins" "--message-format=json" "--release" "--target" "bpfel-unknown-none" "--target-dir" "/Users/lielfridman/Projects/Rust/ebpf-auditer/target/aarch64-unknown-linux-musl/release/build/ebpf-auditer-af3b67f9e39190a6/out/ebpf-auditer-ebpf" failed: ExitStatus(unix_wait_status(25856))
left: Some(101)
right: Some(0)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace |
|
I'll work on adding cross compilation on Mac to our CI. |
|
@lielfr yeah, the instructions were wrong, setting See #131. |
The instructions in the book are now out of date for projects generated
with this template. Since the instructions are much shorter these days,
put them in the README.
Fixes #127.