Issues linking nanobind extension on Windows in Debug mode (VS wants to linke the release lib of Python).

[KerstinKeller]

So we're building Python extensions for our library with nanobind.
Now, since I needed to debug some things, I wanted to build the debug version of the extension.
The debug version of Python is installed and available.
However VS gives me a linker error.
It complains:

1>LINK : fatal error LNK1104: cannot open file 'python312.lib'
1>Done building project "nanobind_core.vcxproj" -- FAILED.
========== Build: 0 succeeded, 1 failed, 5 up-to-date, 0 skipped ==========

First strange thing: it looks for the non-debug lib, even though this is a debug build.
Second strange thing: This linker argument is nowhere to be found in the generated Visual Studio project files!

This is the search output through the project files:

earch "python312" (4 hits in 1 file of 1 searched)
  D:\ecal\build\_cmake_\ecal\lang\python\nanobind_core\src\nanobind_core.vcxproj (4 hits)
	Line 119:       <AdditionalDependencies>Debug\nanobind-staticd.lib;..\..\..\..\ecal\core\Debug\ecal_cored.lib;C:\Users\xxx\AppData\Local\Programs\Python\Python312\libs\python312_d.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib</AdditionalDependencies>
	Line 165:       <AdditionalDependencies>Release\nanobind-static.lib;..\..\..\..\ecal\core\Release\ecal_core.lib;C:\Users\xxx\AppData\Local\Programs\Python\Python312\libs\python312.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib</AdditionalDependencies>
	Line 211:       <AdditionalDependencies>MinSizeRel\nanobind-staticminsize.lib;..\..\..\..\ecal\core\MinSizeRel\ecal_coreminsize.lib;C:\Users\xxx\AppData\Local\Programs\Python\Python312\libs\python312.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib</AdditionalDependencies>
	Line 256:       <AdditionalDependencies>RelWithDebInfo\nanobind-staticreldbg.lib;..\..\..\..\ecal\core\RelWithDebInfo\ecal_corereldbg.lib;C:\Users\xxx\AppData\Local\Programs\Python\Python312\libs\python312.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib</AdditionalDependencies>

I was searching through our code and pybind code for linker pragmas, nothing to be found. So I am officially out of ideas and wanted to ask if anyone has run into the problem or has some other tips where I could take a look.

[wjakob]

I think the import declaration is in the Python headers (#import or #pragma or some MSVC extension). In the nanobind headers, we set a #define that avoids linking against the Python debug library because people usually want to build debug versions of their extension, but with non-debug Python.

[wjakob]

Try setting Py_DEBUG or including Python.h before nanobind.

[KerstinKeller]

Thanks for the quick help.

I am not quite sure that I agree with your line of argumentation, at least that is not what I expected as a user.
Can a Release Python version even load a Debug Python module?

Unfortunately, defining Py_DEBUG does nothing by issuing a ton of macro redefinition warnings in pyconfig.h, but still results in the link error.
I suspect, that nanobind-static also needs to be build accordingly, but doesn't see the Py_DEBUG define if added only to the target created with nanobind_add_module.
If I remove the lines of questions from nb_python.h, the build is working fine. The Py_DEBUG define is not necessary.

In general, I feel that a Windows nanobind_add_module build should work out of the box for both Release and Debug builds, and currently that is not the case when using CMake.
I guess what I would recommend is a build flag that the user can set / pass to nanobind_add_module like (NB_DEBUG_LINKS_RELEASE) which let's the user choose their preferred way of linking Python.
However, you also need to take care about linking Python here since this will already link Python::Module which will certainly link the found debug python version. So it's essential that there is no discrepancy between the linker arguments from Python::Module and autolinking. Maybe CMake can be convinced to link to Release if Debug was not found.

(There is also an open PR to the Python repository opened back in 2020, suggesting a macro to disable autolinking)
(The vcpkg port of pybind experienced a similar issue)

I am happy to make a PR or test samples on Windows.

[ThoreWietzke]

In my experience, using a Release build of Python works with extension modules with Debug symbols. During debugging in C++, you will only stop at your own written code (including code from nanobind).
I experienced no issue during compilation and also during Debugging.

[macdew]

I'm in the same boat as @KerstinKeller, in that my Visual Studio project is failing to build a Debug module with the same error:

LINK : fatal error LNK1104: cannot open file 'python3.lib'

(I'm targeting the Stable API, hence python3.lib instead of 312.lib)

I'm assuming that something in CMake isn't connecting the release settings of FindPython properly to the Debug config of our module target?

[macdew]

Note: we may be getting tripped up by Python itself rather than nanobind.... I found this in the Python headers (pyconfig.h):

/* For an MSVC DLL, we can nominate the .lib files used by extensions */
#ifdef MS_COREDLL
#       if !defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_BUILTIN)
                /* not building the core - must be an ext */
#               if defined(_MSC_VER)
                        /* So MSVC users need not specify the .lib
                        file in their Makefile (other compilers are
                        generally taken care of by distutils.) */
#                       if defined(_DEBUG)
#                               pragma comment(lib,"python312_d.lib")
#                       elif defined(Py_LIMITED_API)
#                               pragma comment(lib,"python3.lib")
#                       else
#                               pragma comment(lib,"python312.lib")
#                       endif /* _DEBUG */
#               endif /* _MSC_VER */
#       endif /* Py_BUILD_CORE */
#endif /* MS_COREDLL */

This code looks blatantly wrong to me: the debug check should allow for Py_LIMITED_API and link to pyton3_d.lib. I can see that my Visual Studio linker settings are correct in Debug, that is indeed trying to link to: python3_d.lib