Superbuild problem with MSVC versions

Looks like that’s not the right thing yet!

Ok, first of all I did not really understand your hint, because I did not see much handling of MSVC versions in either boost.common.cmake or boost.system.cmake.

And second it is clear that the failure happens inside a “pip install” call that is supposed to do some “building” step, using available build tools.

After the “ninja” call of the superbuild terminates with error, inside the target folder I find a folder /superbuild/pybind11/ and inside that subfolders named src, build, stamp and tmp. Inside src, there is a complete set of source code, including a CMakeLists.txt, and inside build there are files that point to an already successfully run cmake.

If now within that build subfolder I look at the defined CMake variables, e.g. with cmake-gui, everything looks perfectly fine! There are CMAKE_ variables defined for the correct compiler, linker and everything that is normal for working with MSVC 2022 in any other project.

Except one “strange” setting: the variable “pc_for_prefix_file” is defined as being “${pcfiledir}/../..“, which looks to me like some variable should have been evaluated and is not. Could that be a hint!? hmm…

If now I do a “configure” and “generate”, then leave cmake-gui and do a “ninja”, I am getting the message that it has nothing left to do. This looks like a dead end road thus.

Now I can also extract the command that is failing and that is:

C:\Windows\system32\cmd.exe /C "cd /D C:\dev\atgview\pack\superbuild\pybind11\src && "C:\Program Files\CMake\bin\cmake.exe" -E env --modify PYTHONPATH=path_list_prepend:C:/dev/atgview/pack/install/Python/Lib/site-packages -- C:/dev/atgview/pack/install/Python/python.exe -m pip install --no-index --no-deps --no-build-isolation --root=C:/dev/atgview/pack/install --prefix=Python C:/dev/atgview/pack/superbuild/pybind11/src && "C:\Program Files\CMake\bin\cmake.exe" -E touch C:/dev/atgview/pack/superbuild/pybind11/stamp/pybind11-pybind11-pip-install"

Also this I can do “manually”: First switch do that indicated folder, and then the next command is that “famous” pip install command. Which tells me this:

C:\dev\atgview\pack\superbuild\pybind11\src>"C:\Program Files\CMake\bin\cmake.exe" -E env --modify PYTHONPATH=path_list_prepend:C:/dev/atgview/pack/install/Python/Lib/site-packages -- C:/dev/atgview/pack/install/Python/python.exe -m pip install --no-index --no-deps --no-build-isolation --root=C:/dev/atgview/pack/install --prefix=Python C:/dev/atgview/pack/superbuild/pybind11/src
Processing c:\dev\atgview\pack\superbuild\pybind11\src
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [48 lines of output]
      -- Building for: Visual Studio 14 2015
      CMake Deprecation Warning at CMakeLists.txt:13 (cmake_minimum_required):
        Compatibility with CMake < 3.10 will be removed from a future version of
        CMake.

        Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
        to tell CMake that the project requires at least <min> but has been updated
        to work with policies introduced by <max> or earlier.


      -- The CXX compiler identification is unknown
      CMake Error at CMakeLists.txt:49 (project):
        No CMAKE_CXX_COMPILER could be found.



      -- Configuring incomplete, errors occurred!
      Traceback (most recent call last):
        File "<string>", line 117, in remove_output
        File "<string>", line 139, in <module>
        File "C:\dev\atgview\pack\install\Python\lib\subprocess.py", line 526, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['cmake', '-S', '.', '-B', 'C:\\Users\\cornelis\\AppData\\Local\\Temp\\tmplgz3od3n', '-DCMAKE_INSTALL_PREFIX=pybind11', '-DBUILD_TESTING=OFF', '-DPYBIND11_NOPYTHON=ON', '-Dprefix_for_pc_file=${pcfiledir}/../../']' returned non-zero exit status 1.

      During handling of the above exception, another exception occurred:

      Traceback (most recent call last):
        File "C:\dev\atgview\pack\install\Python\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\dev\atgview\pack\install\Python\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\dev\atgview\pack\install\Python\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 149, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "C:\dev\atgview\pack\install\Python\Lib\site-packages\setuptools\build_meta.py", line 396, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "C:\dev\atgview\pack\install\Python\Lib\site-packages\setuptools\build_meta.py", line 341, in run_setup
          exec(code, locals())
        File "<string>", line 123, in <module>
        File "C:\dev\atgview\pack\install\Python\lib\contextlib.py", line 153, in __exit__
          self.gen.throw(typ, value, traceback)
        File "<string>", line 120, in remove_output
        File "C:\dev\atgview\pack\install\Python\lib\shutil.py", line 750, in rmtree
          return _rmtree_unsafe(path, onerror)
        File "C:\dev\atgview\pack\install\Python\lib\shutil.py", line 601, in _rmtree_unsafe
          onerror(os.scandir, path, sys.exc_info())
        File "C:\dev\atgview\pack\install\Python\lib\shutil.py", line 598, in _rmtree_unsafe
          with os.scandir(path) as scandir_it:
      FileNotFoundError: [WinError 3] The system cannot find the path specified: 'pybind11/include'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

And here, thus “inside” the pip install, I am getting the error message “No CMAKE_CXX_COMPILER could be found”. But as I already said: it is present in the CMakeCache.txt of the pybuild11 build folder, and it points to the MSVC 2022 compiler!

But a few lines before the strange error message it also says: “Building for: Visual Studio 14 2015“ - and well yes, I do not have these build tools installed. So the problem is: Why does the “pip install” try to find it?

At the end, it is explicitly stated that the problem is not with pip, but comes from the package.

So basically I am turning in circles here, and my guess is that maybe pip has some way to find a “current” set of MS build tools, but MS has again changed some little thing in the way how it can be found. Then it does not find anything, and looking for MSVC 2015 is some kind of “default” - and again no success because indeed not present.

Trying my good luck with asking some “Brave” search AI stuff finally points to the same direction, giving the one or other hint to be tried.

And indeed one thing brought me one step further: Include the following settings in the environment somehow:

set DISTUTILS_USE_SDK=1
set MSSdk=1

With this, “pip install” is supposed to use another method to find MS build tools, and indeed the pybind11 step was now working indeed! Correctly stating “The CXX compiler identification is MSVC 19.44.35224.0“

Now the “superbuild” hangs at the next step, which is pythonpillow.

But I am not going to investigate that one right now, but rather also look after my pillow :slight_smile: