From eaed64e105ed5f37038ffb6cbba5460e8542e1a2 Mon Sep 17 00:00:00 2001 From: Ted Jenks Date: Tue, 14 Nov 2023 12:38:29 +0000 Subject: [PATCH 1/2] correct the docsa --- .github/workflows/velox_tpch_merge.yml | 1 - docs/developers/HowTo.md | 32 ++++++-------------------- docs/developers/NewToGluten.md | 8 ++----- 3 files changed, 9 insertions(+), 32 deletions(-) diff --git a/.github/workflows/velox_tpch_merge.yml b/.github/workflows/velox_tpch_merge.yml index 0434f969fd10..f323e45382b9 100644 --- a/.github/workflows/velox_tpch_merge.yml +++ b/.github/workflows/velox_tpch_merge.yml @@ -31,7 +31,6 @@ on: - 'shims/**' - 'tools/gluten-it/**' - 'tools/gluten-te/**' - - 'ep/build-arrow/**' - 'ep/build-velox/**' - 'cpp/*' - 'cpp/CMake/**' diff --git a/docs/developers/HowTo.md b/docs/developers/HowTo.md index bc81c5deeca1..f3f2483591ed 100644 --- a/docs/developers/HowTo.md +++ b/docs/developers/HowTo.md @@ -41,32 +41,14 @@ To debug C++, you have to generate the example files, the example files consist You can generate the example files by the following steps: -1. get and build Arrow +1. build Velox ``` -cd gluten_home/ep/build-arrow/src -./get_arrow.sh -./build_arrow.sh -``` - -2. get and build Velox -``` -cd gluten_home/ep/build-velox/src -./get_velox.sh -./build_velox.sh --build_type=Debug -``` - -3. compile the CPP -``` -cd gluten_home/cpp -mkdir build -cd build -cmake -DBUILD_VELOX_BACKEND=ON -DBUILD_TESTS=ON -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Debug .. -make -j +gluten_home/dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --build_type=Debug ``` - Compiling with `--build_type=Debug` is good for debugging. - The executable file `generic_benchmark` will be generated under the directory of `gluten_home/cpp/build/velox/benchmarks/`. -4. build Gluten and generate the example files +2. build Gluten and generate the example files ``` cd gluten_home mvn clean package -Pspark-3.2 -Pbackends-velox -Prss @@ -87,7 +69,7 @@ gluten_home/backends-velox/generated-native-benchmark/ └── _SUCCESS ``` -5. now, run benchmarks with GDB +3. now, run benchmarks with GDB ``` cd gluten_home/cpp/build/velox/benchmarks/ gdb generic_benchmark @@ -99,15 +81,15 @@ gdb generic_benchmark - Actually, you can debug `generic_benchmark` with any gdb commands as debugging normal C++ program, because the `generic_benchmark` is a pure C++ executable file in fact. -6. `gdb-tui` is a valuable feature and is worth trying. You can get more help from the online docs. +4. `gdb-tui` is a valuable feature and is worth trying. You can get more help from the online docs. [gdb-tui](https://sourceware.org/gdb/onlinedocs/gdb/TUI.html) -7. you can start `generic_benchmark` with specific JSON plan and input files +5. you can start `generic_benchmark` with specific JSON plan and input files - If you omit them, the `example.json, example_lineitem + example_orders` under the directory of `gluten_home/backends-velox/generated-native-benchmark` will be used as default. - You can also edit the file `example.json` to custom the Substrait plan or specify the inputs files placed in the other directory. -8. get more detail information about benchmarks from [MicroBenchmarks](./MicroBenchmarks.md) +6. get more detail information about benchmarks from [MicroBenchmarks](./MicroBenchmarks.md) ## 2 How to debug Java/Scala wait to add diff --git a/docs/developers/NewToGluten.md b/docs/developers/NewToGluten.md index 64e5441d18ac..38e88c1195e9 100644 --- a/docs/developers/NewToGluten.md +++ b/docs/developers/NewToGluten.md @@ -58,12 +58,8 @@ You can just refer to [build-gluten-with-velox-backend](../get-started/Velox.md# If you need to debug cpp code, please compile the backend code and gluten cpp code with debug mode. ```bash -## compile velox -./build_velox.sh --build_type=Debug -## compile arrow with tests required library -./build_arrow.sh -## compile gluten cpp with benchmark and tests to debug -cmake -DBUILD_VELOX_BACKEND=ON -DBUILD_TESTS=ON -DBUILD_BENCHMARKS=ON -DCMAKE_BUILD_TYPE=Debug .. +## compile velox backend with benchmark and tests to debug +gluten_home/dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --build_type=Debug ``` If you need to debug the tests in /gluten-ut, You need to compile java code with `-P spark-ut`. From 5b3fe9e765a1bdd717353eadbde797dbd4b28a78 Mon Sep 17 00:00:00 2001 From: Ted Jenks Date: Wed, 15 Nov 2023 11:25:57 +0000 Subject: [PATCH 2/2] correct docs --- docs/developers/HowTo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developers/HowTo.md b/docs/developers/HowTo.md index f3f2483591ed..e4bab4a26883 100644 --- a/docs/developers/HowTo.md +++ b/docs/developers/HowTo.md @@ -41,7 +41,7 @@ To debug C++, you have to generate the example files, the example files consist You can generate the example files by the following steps: -1. build Velox +1. build Velox and Gluten CPP ``` gluten_home/dev/builddeps-veloxbe.sh --build_tests=ON --build_benchmarks=ON --build_type=Debug ```