llama : reorganize source code + improve CMake

[ggerganov]

ref #7573, #6913

Adopt a new structure of the source code that makes ggml source code and build scripts easier to reuse.

Note that the build options that are relevant to the ggml library are now prefixed with GGML_. For example, LLAMA_CUDA and WHISPER_METAL are now GGML_CUDA and GGML_METAL. However, WHISPER_COREML and WHISPER_OPENVINO are still named the same because the CoreML and OpenVINO functionality in whisper.cpp is not part of the ggml library.

The Makefiles in llama.cpp and whisper.cpp have been updated to be more similar with each other.

Header files (such as ggml.h, llama.h and whisper.h) are now placed in include subfolders, while the source files are placed in src.

PRs in other projects that will be updated and merged together with this one:

• ggml : reorganize source code + improve CMake ggml#865
• whisper : reorganize source code + improve CMake whisper.cpp#2256

TODOs:

• update ggml repo
• update whisper.cpp repo
• update Makefiles
  • llama.cpp
  • whisper.cpp
• update sync scripts
  • llama.cpp
  • ggml
  • whisper.cpp
  • add ggml/cmake
• fix examples
• fix CI
  • HIPBLAS
  • kompute
  • nix (maybe will work after merge to master?)
  • windows
• update README with new GGML_ options
  • llama.cpp
  • whisper.cpp

Changes:

• deprecate LLAMA_XXX in favor of GGML_XXX

Resolved PRs:

• [SYCL] Fix windows build and inference #8003
• ggml : remove ggml_task_type and GGML_PERF #8017
• CUDA: use MMQ instead of cuBLAS by default #8075

TODO in follow-up PRs:

• move relevant tests from tests to ggml/tests
• avoid using CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS and build and link shared libraries properly
• link shared libs to examples by default
• simplify sync scripts to work on the ggml folder level

---

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[ggerganov]

The llama.cpp reorganization should be now mostly ready and can be tested. The ggml code, tests and cmake scripts will be shared across all 3 repos (as if it is a git submodule).

Some CI workflows are still failing - any help with resolving these is appreciated. I'll now focus on updating whisper.cpp to reuse ggml in a similar way and then look to merge this sometime next week. Extra attention to the new build options (e.g. LLAMA_CUDA is now GGML_CUDA)

[slaren]

It works with -DBUILD_SHARED_LIBS=OFF, so I am probably misunderstanding the error.

[slaren]

I noticed now that CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS is set in ggml when building on windows with shared libs. I think that the problem actually is that CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS is not set early enough, because configuring with -DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON also works, without requiring the changes that I listed before.

This seems to work. My assumption is that CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS needs to be set per project, unless passed through the command line, then it is applied to every subproject as well, but I am not sure if that's correct.

However, all of this seems like a hack, we go through the effort to use dllexport in ggml.h and llama.h with all the symbols that should be exported, but then we just throw all of this away and use CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS to export every symbol.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 6b9b5413..96718d75 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -6,6 +6,7 @@ include(CheckIncludeFileCXX)
 set(CMAKE_WARN_UNUSED_CLI YES)

 set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
+set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)

 if (NOT XCODE AND NOT MSVC AND NOT CMAKE_BUILD_TYPE)
     set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
diff --git a/common/CMakeLists.txt b/common/CMakeLists.txt
index c6fccc02..02415f2d 100644
--- a/common/CMakeLists.txt
+++ b/common/CMakeLists.txt
@@ -1,5 +1,7 @@
 # common

+set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
+
 find_package(Threads REQUIRED)

 # Build info header
diff --git a/ggml/CMakeLists.txt b/ggml/CMakeLists.txt
index bdbda425..21ef7e4a 100644
--- a/ggml/CMakeLists.txt
+++ b/ggml/CMakeLists.txt
@@ -3,6 +3,7 @@ project("ggml" C CXX)
 include(CheckIncludeFileCXX)

 set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
+set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)

 if (NOT XCODE AND NOT MSVC AND NOT CMAKE_BUILD_TYPE)
     set(CMAKE_BUILD_TYPE Release CACHE STRING "Build type" FORCE)
diff --git a/ggml/src/CMakeLists.txt b/ggml/src/CMakeLists.txt
index 84bc8e19..b7c79321 100644
--- a/ggml/src/CMakeLists.txt
+++ b/ggml/src/CMakeLists.txt
@@ -825,10 +825,6 @@ endif()

 if (WIN32)
     add_compile_definitions(_CRT_SECURE_NO_WARNINGS)
-
-    if (BUILD_SHARED_LIBS)
-        set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
-    endif()
 endif()

 if (GGML_LTO)

[slaren]

GGML_BUILD_SHARED_LIBS and LLAMA_BUILD_SHARED_LIBS do not exist, BUILD_SHARED_LIBS is used directly.

[mofosyne]

if you are reorganizing source codes, I would like to suggest moving some compiled tools from the example folder into a dedicated folder. It's not quite a script, but in my opinion its not examples as well... considering it's more for maintainers internal usage

[OuadiElfarouki]

@slaren The LLAMA_CUDA_FORCE_CUBLAS cmake option got mistakenly removed but is still used. I believe it's intended to be mutually exclusive with GGML_CUDA_FORCE_MMQ so some changes might be needed (cmake, mmq.cu, ggml-cuda.cu ..)

[ggerganov]

There is now GGML_CUDA_FORCE_CUBLAS

Edit: nvm #8140