openmp-extractor is broken

[chunhualiao]

http://freecompilercamp.org/openmp-extractor/

It seems like the default container uses llvm 10.x while the tutorial was developed to work with the older llvm.

Can you prepare an old version llvm container to restore this tutorial?

You may want to check which old version of llvm we used before.

[chunhualiao]

Alternative, you can update the tutorial to work with llvm 10.x

the source codes are imported into https://github.com/freeCompilerCamp/code-for-llvm-tutorials/tree/master/OpenMP-Extractor

It works on my local machine, using latest llvm:

   62  git clone https://github.com/freeCompilerCamp/code-for-llvm-tutorials.git
   63  mkdir code-for-llvm-tutorials/OpenMP-Extractor/build
   64  cd code-for-llvm-tutorials/OpenMP-Extractor/build
   67  cmake ../.
   68  make
   69  cat << EOF > test.c
#include <stdlib.h>
#include <stdio.h>
int main (void)
{
  int sum=0;
  #pragma omp parallel for reduction(+:sum)
  for (int i = 0; i < 100; i++)
  {
    sum += 1;
  }
  printf ("sum = %d\n",sum);
  return 0;
}
EOF

   70  clang -Xclang -load -Xclang ./ompextractor/libCLANGOMPExtractor.so -Xclang -add-plugin -Xclang -extract-omp -fopenmp -g -O0 -c -fsyntax-only test.c
   71  ls
   72  vi test.c.json 

[chunhualiao]

the execution has some error in freeCC's terminal

freecc@node1:build$ clang -Xclang -load -Xclang ./ompextractor/libCLANGOMPExtractor.so -Xclang -add-plugin -Xclang -extract-omp -fopenmp -g -O0 -c -fsyntax-only test.c
Stack dump:
0.      Program arguments: clang -Xclang -load -Xclang ./ompextractor/libCLANGOMPExtractor.so -Xclang -add-plugin -Xclang -extract-omp -fopenmp -g -O0 -c -fsyntax-only test.c
1.      <eof> parser at end of file
 #0 0x000055682f77e0aa llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/home/freecc/install/llvm_install/bin/clang-10+0x288a0aa)
 #1 0x000055682f77bc94 llvm::sys::RunSignalHandlers() (/home/freecc/install/llvm_install/bin/clang-10+0x2887c94)
 #2 0x000055682f77bf05 llvm::sys::CleanupOnSignal(unsigned long) (/home/freecc/install/llvm_install/bin/clang-10+0x2887f05)
 #3 0x000055682f6f92f0 CrashRecoverySignalHandler(int) (/home/freecc/install/llvm_install/bin/clang-10+0x28052f0)
 #4 0x00007ff4d854e890 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12890)
 #5 0x000055683199c445 clang::Stmt::IgnoreContainers(bool) (/home/freecc/install/llvm_install/bin/clang-10+0x4aa8445)
 #6 0x00007ff4d70f4f2a PragmaVisitor::visitNodes(clang::Stmt*, std::vector<clang::Stmt*, std::allocator<clang::Stmt*> >&) (./ompextractor/libCLANGOMPExtractor.so+0x16ef2a)
 #7 0x00007ff4d70f4f43 PragmaVisitor::visitNodes(clang::Stmt*, std::vector<clang::Stmt*, std::allocator<clang::Stmt*> >&) (./ompextractor/libCLANGOMPExtractor.so+0x16ef43)
 #8 0x00007ff4d70f4f43 PragmaVisitor::visitNodes(clang::Stmt*, std::vector<clang::Stmt*, std::allocator<clang::Stmt*> >&) (./ompextractor/libCLANGOMPExtractor.so+0x16ef43)
 #9 0x00007ff4d70fb314 PragmaVisitor::VisitDecl(clang::Decl*) (./ompextractor/libCLANGOMPExtractor.so+0x175314)
#10 0x00007ff4d712f554 clang::RecursiveASTVisitor<PragmaVisitor>::WalkUpFromDecl(clang::Decl*) (./ompextractor/libCLANGOMPExtractor.so+0x1a9554)
#11 0x00007ff4d71447b4 clang::RecursiveASTVisitor<PragmaVisitor>::WalkUpFromNamedDecl(clang::NamedDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x1be7b4)
#12 0x00007ff4d7148bda clang::RecursiveASTVisitor<PragmaVisitor>::WalkUpFromValueDecl(clang::ValueDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x1c2bda)
#13 0x00007ff4d7148c92 clang::RecursiveASTVisitor<PragmaVisitor>::WalkUpFromDeclaratorDecl(clang::DeclaratorDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x1c2c92)
#14 0x00007ff4d7126b26 clang::RecursiveASTVisitor<PragmaVisitor>::WalkUpFromFunctionDecl(clang::FunctionDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x1a0b26)
#15 0x00007ff4d711361d clang::RecursiveASTVisitor<PragmaVisitor>::TraverseFunctionDecl(clang::FunctionDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x18d61d)
#16 0x00007ff4d7103a64 clang::RecursiveASTVisitor<PragmaVisitor>::TraverseDecl(clang::Decl*) (./ompextractor/libCLANGOMPExtractor.so+0x17da64)
#17 0x00007ff4d711f368 clang::RecursiveASTVisitor<PragmaVisitor>::TraverseDeclContextHelper(clang::DeclContext*) (./ompextractor/libCLANGOMPExtractor.so+0x199368)
#18 0x00007ff4d711710a clang::RecursiveASTVisitor<PragmaVisitor>::TraverseTranslationUnitDecl(clang::TranslationUnitDecl*) (./ompextractor/libCLANGOMPExtractor.so+0x19110a)
#19 0x00007ff4d7103ff8 clang::RecursiveASTVisitor<PragmaVisitor>::TraverseDecl(clang::Decl*) (./ompextractor/libCLANGOMPExtractor.so+0x17dff8)
#20 0x00007ff4d70fbc16 PragmaASTConsumer::HandleTranslationUnit(clang::ASTContext&) (./ompextractor/libCLANGOMPExtractor.so+0x175c16)
#21 0x000055682ff16668 clang::MultiplexConsumer::HandleTranslationUnit(clang::ASTContext&) (/home/freecc/install/llvm_install/bin/clang-10+0x3022668)
#22 0x0000556830e8c9b9 clang::ParseAST(clang::Sema&, bool, bool) (/home/freecc/install/llvm_install/bin/clang-10+0x3f989b9)
#23 0x000055682fedf119 clang::FrontendAction::Execute() (/home/freecc/install/llvm_install/bin/clang-10+0x2feb119)
#24 0x000055682fe9e49a clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/freecc/install/llvm_install/bin/clang-10+0x2faa49a)
#25 0x000055682ffa827b clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/freecc/install/llvm_install/bin/clang-10+0x30b427b)
#26 0x000055682db4e984 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/freecc/install/llvm_install/bin/clang-10+0xc5a984)
#27 0x000055682db4c01d ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&) (/home/freecc/install/llvm_install/bin/clang-10+0xc5801d)
#28 0x000055682fd80fb5 void llvm::function_ref<void ()>::callback_fn<clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const::'lambda'()>(long) (/home/freecc/install/llvm_install/bin/clang-10+0x2e8cfb5)
#29 0x000055682f6f93d3 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) (/home/freecc/install/llvm_install/bin/clang-10+0x28053d3)
#30 0x000055682fd81c20 clang::driver::CC1Command::Execute(llvm::ArrayRef<llvm::Optional<llvm::StringRef> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, bool*) const (.part.148) (/home/freecc/install/llvm_install/bin/clang-10+0x2e8dc20)
#31 0x000055682fd5bdb5 clang::driver::Compilation::ExecuteCommand(clang::driver::Command const&, clang::driver::Command const*&) const (/home/freecc/install/llvm_install/bin/clang-10+0x2e67db5)
#32 0x000055682fd5c811 clang::driver::Compilation::ExecuteJobs(clang::driver::JobList const&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) const (/home/freecc/install/llvm_install/bin/clang-10+0x2e68811)
#33 0x000055682fd65405 clang::driver::Driver::ExecuteCompilation(clang::driver::Compilation&, llvm::SmallVectorImpl<std::pair<int, clang::driver::Command const*> >&) (/home/freecc/install/llvm_install/bin/clang-10+0x2e71405)
#34 0x000055682daba5fc main (/home/freecc/install/llvm_install/bin/clang-10+0xbc65fc)
#35 0x00007ff4d740cb97 __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:344:0
#36 0x000055682db4bb9a _start (/home/freecc/install/llvm_install/bin/clang-10+0xc57b9a)
clang-10: error: clang frontend command failed due to signal (use -v to seeinvocation)
clang version 10.0.1 (https://github.com/llvm/llvm-project 6f71678ecd293c35435f7f3f313bdaf337ed798a)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/freecc/install/llvm_install/bin
clang-10: note: diagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
clang-10: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-10: note: diagnostic msg: /tmp/test-5fc00c.c
clang-10: note: diagnostic msg: /tmp/test-5fc00c.sh
clang-10: note: diagnostic msg:

********************

freecc@node1:llvm_src$ git show 6f71678
commit 6f71678ecd293c35435f7f3f313bdaf337ed798a (HEAD -> release/10.x, origin/release/10.x)
Author: serge-sans-paille <[email protected]>
Date:   Mon May 4 08:06:39 2020 +0200

    Make Polly tests dependencies explicit

    Due to libPolly now using the component infrastructure, it no longer carries all
    dependencies as it used to do.

    Differential Revision: https://reviews.llvm.org/D79295

    (cherry picked from commit 8ceee08de135c4c96980bd750bb5e95348126980)

diff --git a/polly/unittests/ScopPassManager/CMakeLists.txt b/polly/unittests/ScopPassManager/CMakeLists.txt
index 59c5a4a717a..ed3bbd6dcef 100644
--- a/polly/unittests/ScopPassManager/CMakeLists.txt
+++ b/polly/unittests/ScopPassManager/CMakeLists.txt
@@ -1,3 +1,5 @@
+llvm_map_components_to_libnames(llvm_libs Passes Core Analysis)
 add_polly_unittest(ScopPassManagerTests
   PassManagerTest.cpp
   )
+target_link_libraries(ScopPassManagerTests PRIVATE ${llvm_libs})

[chunhualiao]

I built a fresh LLVM off github's 10.x today . It works

   70  clang -Xclang -load -Xclang ./ompextractor/libCLANGOMPExtractor.so -Xclang -add-plugin -Xclang -extract-omp -fopenmp -g -O0 -c -fsyntax-only test.c
   71  ls
   72  vi test.c.json 
   73  history
ubuntu@ip-128-115-246-11:~/code-for-llvm-tutorials/OpenMP-Extractor/build$ clang --version
clang version 10.0.1 (https://github.com/llvm/llvm-project d24d5c8e308e689dcd83cbafd2e8bd32aa845a15)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/ubuntu/install/llvm_install/bin

ubuntu@ip-128-115-246-11:~/source/llvm_src$ git show d24d5c8e308e689dcd83cbafd2e8bd32aa845a15
commit d24d5c8e308e689dcd83cbafd2e8bd32aa845a15 (HEAD -> release/10.x, origin/release/10.x)
Author: Scott Constable <[email protected]>
Date:   Wed Jun 17 20:02:08 2020 -0700

    fixed broken test after cherry pick

diff --git a/llvm/test/CodeGen/X86/lvi-hardening-loads.ll b/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
index e40911aa1f0..70eebcf0d66 100644
--- a/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
+++ b/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
@@ -111,9 +111,9 @@ if.then:                                          ; preds = %for.body
 ; X64-NOOPT-NEXT:      lfence
 ; X64-NOOPT-NEXT:      movq (%rax,%rcx,8), %rax
 ; X64-NOOPT-NEXT:      lfence
-; X64-NOOPT-NEXT:      movl (%rax), %eax
+; X64-NOOPT-NEXT:      movl (%rax), %edx
 ; X64-NOOPT-NEXT:      lfence
-; X64-NOOPT-NEXT:      movl %eax, -{{[0-9]+}}(%rsp)
+; X64-NOOPT-NEXT:      movl %edx, -{{[0-9]+}}(%rsp)
 
 if.end:                                           ; preds = %if.then, %for.body
   br label %for.inc

[chunhualiao]

@ouankou It seems that the verison of llvm our terminal uses (May 4th) has some bug. Later version (june 17th) fixed it. Can you update llvm?

[ouankou]

@chunhualiao For now, in that tutorial I specified that it uses our initial docker image with LLVM 8.x on Ubuntu 16.04 (freeCompilerCamp/freecompilercamp.github.io@c662a75).
In the guide of creating new tutorial, I also added this legacy image to the list in case someone need it (freeCompilerCamp/freecompilercamp.github.io@61796a1).

I'll check your new code and update the tutorial to work with LLVM 10.x. The new docker images are being built, which will take a few hours.

[ouankou]

@chunhualiao I've pushed the latest LLVM 10.x build to FreeCC, which is the exactly same version of your local LLVM build. However, it still throws the same error in the FreeCC terminal.
I also compiled the latest LLVM 10.x locally and got the same error.

Would you mind sharing your compilation parameters for building LLVM? What I use is:

...
cmake -G Ninja -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$LLVM_PATH -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;compiler-rt;openmp" -DCLANG_BUILD_EXAMPLES=1 $LLVM_SRC/llvm
ninja install -j8 -l8