Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new(driver): add ppc64le support to old bpf and kmod plus CI job #1497

Merged
merged 12 commits into from
Nov 29, 2023
Merged

new(driver): add ppc64le support to old bpf and kmod plus CI job #1497

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9dd72f5
Add checks to support kmod,bpf on ppc64le
mdafsanhossain Nov 21, 2023
5d012ef
CI: build libs kmod,bpf,scap,unit-test-libsinsp on ppc64le
mdafsanhossain Nov 21, 2023
737247e
paths-filter for ppc64le build libs job
mdafsanhossain Nov 21, 2023
1b38114
new(bpf):Adds PPC64 config for BPF_SUPPORTS_RAW_TRACEPOINTS
mdafsanhossain Nov 24, 2023
7a3b8af
fix:remove extra parentheses
mdafsanhossain Nov 24, 2023
201397d
new(kmod):PPC64 fixes for ia32_syscall
mdafsanhossain Nov 27, 2023
f57237d
fix(test_suite): execveat_x fix for powerpc
mdafsanhossain Nov 28, 2023
32fe4a9
new(CI):Combine s390x,ppc64le build jobs and remove libsinsp-unit-test
mdafsanhossain Nov 29, 2023
c61230f
new(CI): Add paths-filter to s390x/ppc64le job
mdafsanhossain Nov 29, 2023
28a356d
fix: Rename powerpc macro and include riscv64 syscall headers
mdafsanhossain Nov 29, 2023
2056fb8
CI:Remove scap-open; build drivers_test for s390x/ppc64le
mdafsanhossain Nov 29, 2023
5079dae
CI:Set fail-fast to false and change s390x/ppc64le job name
mdafsanhossain Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 15 additions & 11 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,11 +323,17 @@ jobs:
cd build
sudo ./test/drivers/drivers_test -k

build-libs-s390x:
name: build-libs-s390x 😁 (system_deps)
build-drivers-s390x-ppc64le:
name: build-drivers-${{ matrix.arch }} 😁 (system_deps)
runs-on: ubuntu-22.04
needs: paths-filter
if: needs.paths-filter.outputs.driver_changed == 'true' || needs.paths-filter.outputs.libsinsp_changed == 'true'
if: needs.paths-filter.outputs.driver_changed == 'true' || needs.paths-filter.outputs.libscap_changed == 'true'
strategy:
matrix:
arch: [s390x, ppc64le]
fail-fast: false
env:
PLATFORM: ${{ matrix.arch == 'ppc64le' && 'powerpc64le' || 's390x' }}
steps:

- name: Checkout Libs ⤵️
Expand All @@ -336,31 +342,29 @@ jobs:
fetch-depth: 0

- uses: uraimo/run-on-arch-action@4ed76f16f09d12e83abd8a49e1ac1e5bf08784d4 # v2.5.1
name: Run s390x build 🏗️
name: Run ${{ matrix.arch }} build 🏗️
with:
arch: s390x
arch: ${{ matrix.arch }}
distro: ubuntu22.04
githubToken: ${{ github.token }}

install: |
apt update && apt install -y --no-install-recommends ca-certificates cmake build-essential clang llvm git pkg-config autoconf automake libtool libelf-dev wget libc-ares-dev libcurl4-openssl-dev libssl-dev libtbb-dev libjq-dev libjsoncpp-dev libgrpc++-dev protobuf-compiler-grpc libgtest-dev libprotobuf-dev linux-headers-generic
apt update && apt install -y --no-install-recommends ca-certificates cmake build-essential clang llvm git pkg-config autoconf automake libtool libelf-dev wget libc-ares-dev libcurl4-openssl-dev libssl-dev libtbb-dev libjq-dev libjsoncpp-dev libgrpc++-dev protobuf-compiler-grpc libcap-dev libgtest-dev libprotobuf-dev linux-headers-generic
git clone https://github.com/libbpf/bpftool.git --branch v7.0.0 --single-branch
cd bpftool
git submodule update --init
cd src && make install
cd ../../
git clone https://github.com/libbpf/libbpf.git --branch v1.0.1 --single-branch
cd libbpf/src && BUILD_STATIC_ONLY=y DESTDIR=/ make install
ln -s /usr/lib64/libbpf.a /usr/lib/s390x-linux-gnu/

ln -s /usr/lib64/libbpf.a /usr/lib/${{env.PLATFORM}}-linux-gnu/
# Please note: we cannot inject the BPF probe inside QEMU, so right now, we only build it
run: |
git config --global --add safe.directory $GITHUB_WORKSPACE
.github/install-deps.sh
mkdir -p build
cd build && cmake -DBUILD_BPF=On -DUSE_BUNDLED_DEPS=OFF -DMODERN_PROBE_INCLUDE="-I/usr/include/s390x-linux-gnu" -DBUILD_LIBSCAP_MODERN_BPF=ON -DMODERN_BPF_DEBUG_MODE=ON -DBUILD_LIBSCAP_GVISOR=OFF ../
KERNELDIR=/lib/modules/$(ls /lib/modules)/build make scap-open driver bpf unit-test-libsinsp -j6
./libsinsp/test/unit-test-libsinsp
cd build && cmake -DBUILD_BPF=On -DUSE_BUNDLED_DEPS=OFF -DMODERN_PROBE_INCLUDE="-I/usr/include/${{env.PLATFORM}}-linux-gnu" -DBUILD_LIBSCAP_MODERN_BPF=ON -DMODERN_BPF_DEBUG_MODE=ON -DENABLE_DRIVERS_TESTS=On -DCREATE_TEST_TARGETS=On -DBUILD_LIBSCAP_GVISOR=OFF ../
KERNELDIR=/lib/modules/$(ls /lib/modules)/build make driver bpf drivers_test -j6

# This job checks that a bundled deps of libs is as static as possible
test-libs-static:
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Our drivers officially support the following architectures:
| **aarch64** | >= [3.16](https://github.com/torvalds/linux/commit/055b1212d141f1f398fca548f8147787c0b6253f) | >= 4.17 | >= 5.8 | _STABLE_ |
| **s390x** | >= 2.6 | >= [5.5](https://github.com/torvalds/linux/commit/6ae08ae3dea) | >= 5.8 | _EXPERIMENTAL_ |
| **riscv64** | >= [5.0](https://github.com/torvalds/linux/commit/5aeb1b36cedd3a1dfdbfe368629fed52dee34103) | N/A | N/A | _EXPERIMENTAL_ |
| **ppc64le** | N/A | N/A | >= 5.8 | _EXPERIMENTAL_ |
| **ppc64le** | >=2.6 | >=4.18 | >= 5.8 | _EXPERIMENTAL_ |


To access up-to-date status reports on Falco drivers kernel testing, please visit this [page](https://falcosecurity.github.io/libs/). It provides a list of supported syscalls as well as the [report](https://falcosecurity.github.io/libs/report/).
Expand Down
4 changes: 3 additions & 1 deletion driver/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ set(TARGET_ARCH ${CMAKE_HOST_SYSTEM_PROCESSOR})
if((NOT TARGET_ARCH STREQUAL "x86_64") AND
(NOT TARGET_ARCH STREQUAL "aarch64") AND
(NOT TARGET_ARCH STREQUAL "s390x") AND
(NOT TARGET_ARCH STREQUAL "riscv64"))
(NOT TARGET_ARCH STREQUAL "riscv64") AND
(NOT TARGET_ARCH STREQUAL "ppc64le"))
message(WARNING "Target architecture not officially supported by our drivers!")
else()
# Load current kernel version
Expand All @@ -26,6 +27,7 @@ else()
set(kmod_min_kver_map_aarch64 3.16)
set(kmod_min_kver_map_s390x 2.6)
set(kmod_min_kver_map_riscv64 5.0)
set(kmod_min_kver_map_ppc64le 2.6)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like for bpf, kmod too needs some small work:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FedeDP I was going through this

libs/driver/feature_gates.h

Line 64 in c2fd308

#if defined(CONFIG_ARM64) || defined(CONFIG_S390) || defined(CONFIG_RISCV)

We need this patch for ppc64 too?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure! That depends if ppc64le is hit by the kernel behavior of not sending the clone exit event for the child; at the moment, all supported architectures but x86_64 are hit.
We have drivers tests for it though, eg: https://github.com/falcosecurity/libs/blob/master/test/drivers/test_suites/syscall_exit_suite/clone3_x.cpp#L191
So, if this test is passing fine running on ppc64le, we should not need the CAPTURE_SCHED_PROC_FORK enabled.
The same goes for CAPTURE_SCHED_PROC_EXEC.

cc @Andreagit97

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uhm it seems like ppc64le behaves like x86 since looking at failed tests here #1475 (comment) execve and clone test are passing.

BTW always looking at the tests there is another issue:

[  FAILED  ] SyscallExit.execveatX_correct_exit
[  FAILED  ] SyscallExit.execveatX_execve_exit
[  FAILED  ] SyscallExit.execveatX_success_memfd

it seems that ppc64le correctly returns an execveat exit event when an execveat syscall is called (like s390x and riscv)

libs/test/drivers/test_suites/syscall_exit_suite/execveat_x.cpp

Line 214 in c2fd308

#if defined(__s390x__) || defined(__riscv)

Other architectures like x86 instead return an execve instead of an execveat exit event.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @FedeDP @Andreagit97. I reran the test and yes clone3_x is passing so we won't need the patch.

[ RUN      ] SyscallExit.clone3X_father
[       OK ] SyscallExit.clone3X_father (2780 ms)
[ RUN      ] SyscallExit.clone3X_child
[       OK ] SyscallExit.clone3X_child (1139 ms)
[ RUN      ] SyscallExit.clone3X_create_child_with_2_threads
[       OK ] SyscallExit.clone3X_create_child_with_2_threads (1659 ms)
[ RUN      ] SyscallExit.clone3X_child_clone_parent_flag
[       OK ] SyscallExit.clone3X_child_clone_parent_flag (899 ms)
[ RUN      ] SyscallExit.clone3X_child_new_namespace_from_child
[       OK ] SyscallExit.clone3X_child_new_namespace_from_child (1389 ms)
[ RUN      ] SyscallExit.clone3X_child_new_namespace_from_caller
[       OK ] SyscallExit.clone3X_child_new_namespace_from_caller (3079 ms)
[ RUN      ] SyscallExit.clone3X_child_new_namespace_create_thread
[       OK ] SyscallExit.clone3X_child_new_namespace_create_thread (2649 ms)
[ RUN      ] SyscallExit.cloneX_father
[       OK ] SyscallExit.cloneX_father (2619 ms)
[ RUN      ] SyscallExit.cloneX_child
[       OK ] SyscallExit.cloneX_child (1279 ms)

For the execveat , I will add a condition for ppc64le but should that be done with a separate PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried it. That seems to be the offending line

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the extensive tests!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhm it seems strange

	struct list_head *head = &(task->children);
	struct list_head *next_child = (struct list_head *)_READ(head->next);
	if(next_child != head)
	{
		/* We have at least one child, so we need a reaper for it */
		reaper_pid = find_new_reaper_pid(data, task);
	}

if head!= null, next_child cannot be null because in the worst case, it would be next_child==head, BTW we never dereference next_child so it shouldn't be an issue at all 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep the issue is within find_new_reaper_pid :/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened the issue to track the legacy bpf verifier failure: #1521

if (LINUX_KERNEL_VERSION VERSION_LESS ${kmod_min_kver_map_${TARGET_ARCH}})
message(WARNING "[KMOD] To run this driver you need a Linux kernel version >= ${kmod_min_kver_map_${TARGET_ARCH}} but actual kernel version is: ${UNAME_RESULT}")
endif()
Expand Down
1 change: 1 addition & 0 deletions driver/bpf/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ if(BUILD_BPF)
set(bpf_min_kver_map_x86_64 4.14)
set(bpf_min_kver_map_aarch64 4.17)
set(bpf_min_kver_map_s390x 5.5)
set(bpf_min_kver_map_ppc64le 4.18)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pretty sure that eBPF support will also need same fixes as modern eBPF, ie:

I also noticed that we skipped an include here: https://github.com/falcosecurity/libs/blob/master/userspace/libscap/engine/gvisor/parsers.cpp#L35.
I am not sure whether it is needed, but i guess so given we added aarch64 and s390x too (btw gvisor should only be supported on x86_64 atm, since the option is only available on it: https://github.com/falcosecurity/libs/blob/master/cmake/modules/engine_config.cmake#L26)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. I was able to build bpf and kmod out of the box without any changes, so I assumed I did not need to make any other modifications. I will add the fixes you suggested.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FedeDP Btw, I noticed these 2 lines point to incorrect links for each architecture.

libs/driver/bpf/plumbing_helpers.h

Line 268 in f61e7ff

* https://github.com/libbpf/libbpf/blob/master/src/bpf_tracing.h#L166-L178

libs/driver/bpf/plumbing_helpers.h

Line 289 in f61e7ff

* https://github.com/libbpf/libbpf/blob/master/src/bpf_tracing.h#L132-L144

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think libbpf touched those files (we should've linked a fixed hash instead of master). No problem anyway!

if (LINUX_KERNEL_VERSION VERSION_LESS ${bpf_min_kver_map_${TARGET_ARCH}})
message(WARNING "[BPF] To run this driver you need a Linux kernel version >= ${bpf_min_kver_map_${TARGET_ARCH}} but actual kernel version is: ${UNAME_RESULT}")
endif()
Expand Down
35 changes: 35 additions & 0 deletions driver/bpf/plumbing_helpers.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,14 @@ static __always_inline bool bpf_in_ia32_syscall()
status = _READ(task->thread_info.flags);
return status & _TIF_31BIT;

#elif defined(CONFIG_PPC64)

/* See here for the definition:
* https://github.com/torvalds/linux/blob/9b6de136b5f0158c60844f85286a593cb70fb364/arch/powerpc/include/asm/thread_info.h#L127
*/
status = _READ(task->thread_info.flags);
return status & _TIF_32BIT;

#else

/* Unknown architecture. */
Expand Down Expand Up @@ -172,6 +180,13 @@ static __always_inline long bpf_syscall_get_nr(void *ctx)
id = _READ(regs->int_code);
id = id & 0xffff;

#elif CONFIG_PPC64

/* See here for the definition:
* https://github.com/torvalds/linux/blob/f1a09972a45ae63efbd1587337c4be13b1893330/arch/powerpc/include/asm/syscall.h#L37
*/
id = _READ(regs->gpr[0]);

#endif /* CONFIG_X86_64 */

#else
Expand Down Expand Up @@ -304,6 +319,26 @@ static __always_inline unsigned long bpf_syscall_get_argument_from_ctx(void *ctx
arg = 0;
}

#elif CONFIG_PPC64

/* See here for the definition:
* https://github.com/libbpf/libbpf/blob/master/src/bpf_tracing.h#L290-L306
*/
switch (idx) {
case 0:
arg = _READ(regs->orig_gpr3);
break;
case 1:
case 2:
case 3:
case 4:
case 5:
arg = _READ(regs->gpr[idx+3]);
break;
default:
arg = 0;
}

#endif /* CONFIG_X86_64 */

#else
Expand Down
2 changes: 1 addition & 1 deletion driver/bpf/quirks.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ or GPL2.txt for full copies of the license.
#define BPF_FORBIDS_ZERO_ACCESS
#endif

#if (defined(CONFIG_X86_64) || defined(CONFIG_ARM64) || defined(CONFIG_S390)) && LINUX_VERSION_CODE >= KERNEL_VERSION(4, 17, 0)
#if (defined(CONFIG_X86_64) || defined(CONFIG_ARM64) || defined(CONFIG_S390) || defined(CONFIG_PPC64)) && LINUX_VERSION_CODE >= KERNEL_VERSION(4, 17, 0)
#define BPF_SUPPORTS_RAW_TRACEPOINTS
#endif

Expand Down
3 changes: 3 additions & 0 deletions driver/main.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2133,6 +2133,9 @@ static inline bool kmod_in_ia32_syscall(void)
#elif defined(CONFIG_S390)
if (unlikely(task_thread_info(current)->flags & _TIF_31BIT))
return true;
#elif defined(CONFIG_PPC64)
if (unlikely(task_thread_info(current)->flags & _TIF_32BIT))
return true;
#endif /* CONFIG_X86_64 */
return false;
}
Expand Down
6 changes: 3 additions & 3 deletions test/drivers/test_suites/syscall_exit_suite/execveat_x.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@ TEST(SyscallExit, execveatX_correct_exit)

evt_test->disable_capture();

#if defined(__s390x__) || defined(__riscv)
#if defined(__s390x__) || defined(__riscv) || defined(__powerpc64__)
/* We search for a child event. */
evt_test->assert_event_presence(ret_pid);

Expand Down Expand Up @@ -338,7 +338,7 @@ TEST(SyscallExit, execveatX_execve_exit)
evt_test->disable_capture();

/* `s390x` returns an `EXECVEAT_X` event while other architectures retun an `EXECVE_X` */
#if defined(__s390x__) || defined(__riscv)
#if defined(__s390x__) || defined(__riscv) || defined(__powerpc64__)
/* We search for a child event. */
evt_test->assert_event_absence(ret_pid, PPME_SYSCALL_EXECVE_19_X);
#else
Expand Down Expand Up @@ -494,7 +494,7 @@ TEST(SyscallExit, execveatX_success_memfd)

evt_test->disable_capture();

#if defined(__s390x__) || defined(__riscv)
#if defined(__s390x__) || defined(__riscv) || defined(__powerpc64__)
/* We search for a child event. */
evt_test->assert_event_presence(ret_pid);

Expand Down
4 changes: 4 additions & 0 deletions userspace/libscap/engine/gvisor/parsers.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ limitations under the License.
#include "../../driver/syscall_compat_aarch64.h"
#elif __s390x__
#include "../../driver/syscall_compat_s390x.h"
#elif __powerpc64__
#include "../../driver/syscall_compat_ppc64le.h"
#elif __riscv
#include "../../driver/syscall_compat_riscv64.h"
#endif /* __x86_64__ */

#include <functional>
Expand Down