Provide cmake build system?

[waywardmonkeys]

A cmake build system could possibly replace the autoconf-based build system.

This would be able to generate project files for various IDEs, integrate with something like vcpkg, generates commands.json to make it easier to drive clang-tidy and the static analysis tools, etc.

This could be pretty self-contained and would be fewer files than the autoconf system. Perhaps it could even replace the main / primary build system!

[rptb1]

The MPS primary build system is cc. See "Compiling for production". 😄

The reasons for using autoconf are:

1. make it easy for people who expect ./configure && make as used in many classic FOSS projects
2. to drive CI and automated tools (see design.mps.tests.ci.run.posix)

The configure script outputs a message saying it's not really the thing to use. It's just a front end to the real "system" which is in GNU Make, NMAKE, or Xcodebuild. See "Building the MPS for development".

So we could consider replacing GNU Make, NMAKE, and Xcodebuild with cmake. Autoconf is a red herring.

[lassik]

Not sure whether this is relevant, but cmake is written in C++. Hence a C++ compiler is required to compile cmake for each target platform. (AFAIK cmake must be run on the build platform; it cannot generate portable scripts like GNU automake.) This might make it more difficult to work with some embedded or experimental targets, but perhaps it will not.

[rptb1]

a C++ compiler is required to compile cmake ... more difficult to work with some embedded or experimental targets

This brings up a useful requirement to define.

Whatever "make" we're using needs to be available to developers of the MPS on the "supported" platforms. GNU Make, NMAKE, and Xcodebuild together meet that requirement.

But we can ensure things stay easy for developers porting to platforms that don't have many tools.

From the beginning it's been a design principle that the MPS can be built by a command like:

cc -c -o mps.o -D<something>... <source>.c...

so that we don't start getting tangly nests of dependencies on tools and the MPS stays easy to port.

That in turn means that people should easily be able to cook up some alternative where they need it.

If we introduce a tool like cmake for convenience, keep that build principle, and give a few instructions, then I think things could work out well.

[rptb1]

a design principle that the MPS can be built by a command like

This is defined in detail in design.mps.config.build.cc.

That in turn means that people should easily be able to cook up some alternative where they need it.

And this point is covered in design.mps.config.build.rat, which says:

The simplicity of the build function has also made it easy to set up builds using NMAKE (DOS), MPW (Macintosh), and to get the MPS up and running on other platforms such as FreeBSD and Linux in very little time.

[rptb1]

OK, so kind of interesting, but what do we gain? It seems like a lot of work to switch.

[rptb1]

OK, so kind of interesting, but what do we gain?

@waywardmonkeys you could demonstrate one of the things you mentioned in #146 (comment) ?

CMake only seems to be able to make one target variety (build config) at a time by default.

[waywardmonkeys]

OK, so kind of interesting, but what do we gain?

@waywardmonkeys you could demonstrate one of the things you mentioned in #146 (comment) ?

Yes! I will do so (probably Monday).

CMake only seems to be able to make one target variety (build config) at a time by default.

You can use separate build directories, so this isn't an issue. You can do this either via:

mkdir /some/build/dir
cd /some/build/dir
cmake /path/to/code
cmake --build

or:

cmake /path/to/code -B /some/build/dir
cmake --build /some/build/dir

(You could just run make instead of cmake --build, but what I did will handle different generators and such, so it is easier for something like CI.)

[waywardmonkeys]

So, for clang-tidy for doing static analysis, on your cmake branch, you can do this from the base directory (assuming that like me, you have clang-tidy-15 on your PATH):

cmake code -B build-cmake-clang-tidy -DCMAKE_C_CLANG_TIDY=clang-tidy-15
cmake --build build-cmake-clang-tidy

And then you'll get a ton of output from clang-tidy including some interesting stuff which doesn't belong in this bug. :)

[rptb1]

You can use separate build directories, so this isn't an issue

At the moment, if you ./configure && make test you build and test several varieties (build configurations) and even run comparisons of one against another. The makefile generated by CMake seems only capable of building a single configuration. That means we might have to add extra scripts with loops in them, which themselves won't be Makefiles, and so won't do the dependency checking properly. Our current GNU Make system copes with all that correctly, even when building on several platforms in the same directory.

[rptb1]

extra scripts with loops in them

This sort of thing

cmake . -B hot -DCMAKE_BUILD_TYPE=Release && make -C hot test
cmake . -B cool -DCMAKE_BUILD_TYPE=Debug && make -C cool test

If you can combine -D with --build successfully I would like to see it. CMake's command line parsing is very weird and not Posix-like. I've not been able to combine --build with any other options except --verbose, hence the separate make invocation.

[rptb1]

cmake --build build-cmake-clang-tidy

Woah. Well, it certainly makes a lot of false but interesting claims 😉

[rptb1]

Hmm. https://stackoverflow.com/a/12024211 says:

it appears that CMake has no functionality to detect the target architecture whatsoever.

The horror.

[rptb1]

extra scripts with loops in them

So it seems CMake calls what we do with GNU Make a "multi target config". That is, we build multiple varieties of the same targets with different options. CMake's template using makefiles is only "single configuration". See https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#build-configurations .

I'm not sure how important this is.

[rptb1]

I think the main benefits of CMake for the MPS project could be:

• no need to maintain Xcode and Visual Studio projects (Xcode in particular means developers need access to a macOS desktop)
• no need to maintain parallel NMAKE files to our GNU Make system
• access to extra stuff via CMake generators

[waywardmonkeys]

For cross-compiling, you use toolchain files. Some SDKs / toolchains ship with the appropriate files for their target. It tends to work out pretty well. The fewer checks you do in the cmake config, the better things are of course. (Much like doing fewer things in autoconf as well...)

[rptb1]

The fewer checks you do in the cmake config, the better things are of course

Yes. What I've done so far is to kind of duplicate the way we currently build, without trying to optimise. I bet we could make a pass where we use CMake a bit more "naturally".

[rptb1]

Disturbingly upvoted:

because this is what CMake is about, I guess (a half-documented, moving-target 12-ways-to-do-anything)

This is the feeling I get from reading the documentation and trying to figure out how to do anything. I'm a bit concerned about the dependency.

[rptb1]

The CMake manual says, of CMAKE_SYSTEM_PROCESSOR:

... In many cases, this will correspond to the target architecture for the build, but this is not guaranteed. (E.g. on Windows, the host may be AMD64 even when using a MSVC cl compiler with a 32-bit target.)

Therefore useless.

I'm not sure why they have a variable that might be useful, or might be random junk, and don't even say when it might be useful.

[rptb1]

CheckSymbolExists quietly just doesn't work. e.g.

include(CheckSymbolExists)
check_symbol_exists(MPS_ARCH_I6 mpstd.h MPS_ARCH_I6)
message(VERBOSE "MPS_ARCH_I6 = ${MPS_ARCH_I6}")

I've noticed there's a pattern of "quietly doesn't work". Not great for high reliability software development.

[lassik]

Wow. The time is ripe for a lispmake. The only question is who should make it.

[rptb1]

Here's my current target detection hack. Feels a bit like building on sand.

mps/code/CMakeLists.txt

Lines 47 to 67 in c1d929f

	file(READ mpstd.h FILE_MPSTD_H)
	string(
	REGEX REPLACE "\n[ \t]*#define[ \t]+MPS_PF_STRING[ \t]+" "\n#error "
	FILE_MPSTDE_C "${FILE_MPSTD_H}"
	)
	string(APPEND FILE_MPSTDE_C "int main(void) { return 0; }\n")
	try_compile(
	MPSTDE_RESULT
	SOURCE_FROM_VAR "mpstde.c" FILE_MPSTDE_C
	OUTPUT_VARIABLE MPSTDE_OUTPUT
	)
	if (MPSTDE_OUTPUT MATCHES "#error[ \t]+\"((..)(..)(..))\"")
	set(MPS_PF_STRING ${CMAKE_MATCH_1})
	set(OS ${CMAKE_MATCH_2})
	set(ARCH ${CMAKE_MATCH_3})
	set(COMPILER ${CMAKE_MATCH_4})
	message(VERBOSE "MPS_PF_STRING = ${CMAKE_MATCH_1}")
	message(VERBOSE "OS = ${OS} ARCH = ${ARCH} COMPILER = ${COMPILER}")
	else ()
	message(FATAL_ERROR "Unable to detect target platform")
	endif ()

[rptb1]

I'm not sure why we can't just access cc -dumpmachine. Surely other projects need to know reliably what they're compiling for?

[rptb1]

I've noticed there's a pattern of "quietly doesn't work". Not great for high reliability software development.

cmake . -B foo and cmake -B foo . quietly do different things.

[rptb1]

Before committing to CMake, we should consider other systems. There's a handy list at https://github.com/ninja-build/ninja/wiki/List-of-generators-producing-ninja-build-files

[rptb1]

CMake's template using makefiles is only "single configuration".

I looked in to creating a "multi target config" Makefile generator. Unfortunately, CMake's generators are baked-in C++, such as cmGlobalUnixMakefileGenerator3.cxx. Shame.

[rptb1]

The pull request #148 and its branch demonstrate how CMake might work for the MPS. I think the next steps are:

• review against design.mps.config.req
• list the benefits, such as generating builds for IDEs, cross-compilation, extra tool integrations
• analyse the cost of switching, both short and long term
• weigh up the costs and benefits

An interesting case is #192 which is currently blocked because the i386 platform was dropped from the Xcode project, and I do not have a convenient macOS box to do the necessary merge and validate the result. I can arrange all that and do it, but it's expensive in time.