Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] MSVC: detection fixes and changes #4409

Open
wants to merge 27 commits into
base: master
Choose a base branch
from

Commits on Sep 8, 2023

  1. MSVC: detection fixes and changes

    Changes:
    * VS/VC roots are classified by their installed features (e.g, devenv.com, vcexpress.com, etc.). This provides for special-case verification of batch file arguments based on the internal classification.
    * Consistent with earlier behavior, express versions are used for the non-express msvc version symbol for 14.1 and 8.0 when the express version is the only version installed.
    * There is a strong possibility that 14.0Exp may not have been detected correctly.  It appears that registry keys for earlier versions of msvc are not populated.  Refined detection of 14.0Exp was added.
    * Special case handling of VS2015 BuildTools was added.  The msvc batch files restrict the arguments available as compared to full versions.  The arguments may be accepted and ignored possibly resulting in build failures that are likely not easy to diagnose.
    * Special case handling og VS2015 Express was added.  The msvc batch files restrict the arguments available as compared to full versions.  For example, store/UWP build are only available for x86 targets.
    * Windows/Platform SDK installations of 7.1, 7.0, and 6.1 populate registry keys and installation folders that were detected by scons (versions 10.0 and 9.0).  Unfortunately, the generated files are intended to be used via SetEnv.cmd and result in errors.  The detection of sdk-only installations was added and the roots are ignored.
    * The relative imports of the MSCommon module were changed to top-level absolute imports in a number of microsoft tools.  Moving any of the tools to the site tools folder failed on import (i.e., the relative paths become invalid when moved).
    * VS2005 to VS2015 vcvarsall.bat dispatches to a dependent batch file when configuring the msvc environment.  In certain installation scenarios, the dependent batch file (e.g., vcvars64.bat) may not exist.  The existence of vcvarsall.bat, the dependent batch file, and the compiler executable are now verified.
    * MSVC configuration data specific to versions VS2005 to VS2008 was added as the dependent batch files have different names than the batch files for VS2010 and later.  VC++ For Python is handled as a special case as the dependent batch files: are not used and are in different locations.
    * When VC++ For Python is installed using the ALLUSERS=1 command-line option, the registry keys written are under HKLM rather than HKCU.  VC++ For Python installed for all users is now correctly detected.
    * The existing detection configuration for vswhere and the registry was refactored to separate the two methods of detection.
    * The detection of the msvc compiler executable has been modified and no longer considers the os environment.  The detection of the msvc compiler executable was modified to provide more detailed warning messages.
    jcbrill committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    7d0f804 View commit details
    Browse the repository at this point in the history
  2. Temporarily disable command-line argument and keep original case of v…

    …c path from json output.
    
    Test failures when the vc path from json is normalized.
    Test failure for AddOption help output.
    jcbrill committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    caf8b0f View commit details
    Browse the repository at this point in the history
  3. Update verification of vswhere executable and temporary removal of vs…

    …where command-line argument
    jcbrill committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    1867d37 View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2023

  1. Remove vswhere command-line option and replace retrieval with stub re…

    …turning None. Change processing of user-specified vswhere path.
    
    TODO:
    * revisit MSVC.Util.process_path due to realpath behavior for some windows specifications (drive specifications, etc).
    jcbrill committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    7a9c336 View commit details
    Browse the repository at this point in the history
  2. Update MSCommon/README.rst documentation [ci skip]

    Changes:
    * Added MSVC detection priority.
    * Added VS2015 edition limitations.
    * Updated SDK batch file known issues.
    * Added footnotes for MSVC batch file argument limitations.
    * Added footnote for Windows SDK version numbers (Windows 10 and Windows 11)
    jcbrill committed Sep 10, 2023
    Configuration menu
    Copy the full SHA
    8b4fcde View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3fbef22 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0fc09ba View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    0d37f06 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Configuration menu
    Copy the full SHA
    518fd13 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6424fe3 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0d1aaa7 View commit details
    Browse the repository at this point in the history

Commits on Oct 5, 2023

  1. Merge branch 'master' into jbrill-msvc-fixes

    Manually resolved conflicts in CHANGES.txt and SCons/Tool/MSCommon/vc.py.
    jcbrill committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    fd2ec85 View commit details
    Browse the repository at this point in the history
  2. Internal changes to support channel (preview/release) and component q…

    …ueries in the future.
    
    Internal changes:
    * Rename VISUALSTUDIO_ to MSVS_PRODUCT and vs_def to vs_product_def in all files.
    * Replace process_path with normalize_path and additional arguments in MSCommon.MSVC.Util.
    * Add msvs channel, msvs component id, and msvs component definitions.
    * Add Validate module to MSCommon.MSVC.
    * Add AutoInitialize class to MSCommon.
    * Add custom formatter to MSCommon debug logging to prepend the class name to the function name in the log record based on passed arguments.
    * Rework MSCommon.Kind to return the msvs component, vs directory, vs executable, and vc feature map.
    * Combine _VCVER_TO_PRODUCT_DIR vc version keys into vs product keys and use MSCommon.MSVC.Kind to determine the vs component type.
    * Add file exists and normalized path caches to MSCommon.vc.
    * Rework vswhere executable management and queries.
    * Add an object hierarchy for vs/vc detection. Include preliminary data structure construction for msvs channels (preview, release, any) and component types (Enterprise, Professional, Community, BuildTools, Express, etc).
    * Select an msvc instance rather than a vc dir and pass the instance to MSCommon.MSVC.ScriptArgument methods.
    * Rework msvc action selection (use script, select, use settings, bypass).
    jcbrill committed Oct 5, 2023
    Configuration menu
    Copy the full SHA
    813b8d2 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. Add vs/vc configuration data structures and group related data/functi…

    …ons using internal classes.
    
    Internal changes:
    * Add warning if msvs channel is set after first retrieval and is ignored.
    * Update test case references for internal vc function that returns vswhere executables list.
    * Add temporary code to allow msvs channel and vswhere location to be passed on command-line.
    * Move debug logging for warnings before warn function invocation in case warnings are treated as errors.
    * Add vs/vc configuration objects for detection to class _VSConfig.
    * Move all associated vswhere code to class _VSWhere.
    * Add msvs channel default management class _VSChannel.
    * Move common vs binary/script detection to class _VSCommon.
    * Move msvc instance detection to classes _VSDetectVSWhere and _VSDetectRegistry.
    * Add verification routine for _VSConfig dictionary keys.
    jcbrill committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    38de91a View commit details
    Browse the repository at this point in the history
  2. Remove MSCommon/MSVC/Kind.py

    jcbrill committed Oct 10, 2023
    Configuration menu
    Copy the full SHA
    9b6d7f3 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Update msvs/msvc query implementation.

    Internal changes:
    * Move msvs and msvc instance query methods to appropriate classes.
    * Rework msvc action implementation and logging messages.
    * Move new internal functions with cached return values to class _Util.
    * Add return type annotations for None and bool.
    jcbrill committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    0d68786 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f89eb5a View commit details
    Browse the repository at this point in the history
  3. Move AutoInitialize from common.py to MSVC/Util.py and fix log record…

    … prefix handling for debug output to stdout.
    jcbrill committed Oct 11, 2023
    Configuration menu
    Copy the full SHA
    98f6b3f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    1563ae0 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    5e63e55 View commit details
    Browse the repository at this point in the history

Commits on Oct 12, 2023

  1. New code movement to MSVC modules.

    Internal changes:
    * Move command-line option handling to MSVC/Options.py
    * Move vswhere code to MSVC/VSWhere.py
    * Move new vs detection code to MSVC/VSDetect.py
    * Update test suite accordingly.
    jcbrill committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    98dd971 View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2023

  1. Changes:

    * Log OSError exception for resolve_path in MSCommon/MSVC/Util.py.
    * Fix file docstring for VSDetect.py.
    * Update MSVC action when MSVC_USE_SCRIPT is None and MSVC_USE_SETTINGS is not None (SETTINGS instead of BYPASS).
    * Replace process_path function with resolve_path and normalize_path functions in MSCommon/MSVC/Util.py.
    * Replace process_path invocations with normalize_path invocations.
    * Protect against inadvertent resolve/realpath for windows drive specifications in resolve_path and normalize_path.
    * Additional options for normalize_path with defaults consistent with process_path.
    jcbrill committed Oct 14, 2023
    Configuration menu
    Copy the full SHA
    26bdb85 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7e55fdb View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2023

  1. Merge branch 'master' into jbrill-msvc-channel

    Manually resolved conflicts:
    * CHANGES.txt
    * RELEASE.txt
    jcbrill committed Oct 15, 2023
    Configuration menu
    Copy the full SHA
    2036f17 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Changes:

    * Add env_query function to return the value in a mapping associated with the key, invoking subst method when specified and available. Support general mapping in addition to scons Environments.
    * Add methods that accept env mapping in MSCommon/MSVC/VSDetect.py and MSCommon/MSVC/VSWhere.py.
    jcbrill committed Oct 16, 2023
    Configuration menu
    Copy the full SHA
    e56efcc View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Rework msvc options for running the test suite with preview-only msvc…

    … installations.
    
    Internal changes:
    * Remove "*" as a channel symbol.
    * MSVC options:
      * Use a single environment variable (SCONS_MSVC_OPTIONS).
      * Add an msvc options parser.
      * Always add scons options.
      * Update test/Help.py for msvc local options.
    * MSVC/MSVS detection:
      * Update configuration data for MSVS and update registry keys for MSVC/Config.py.
      * Add msvc exists guard in MSCommon/vcTests.py for sdk list tests in case sdk installed but msvc is not (preview testing).
      * Rework vs.py for msvs instances and temporarily comment-out existing accessor methods and replace with attributes.
    * SCons/Tool/msvsTests.py:
      * Rework DummyExists implementation to accommodate revised msvc/msvs detection.
      * Handle multiple versions returned for express versions.
    * test/Help.py
      * Detect if msvc location options should be present and adjust the expected output accordingly.
    jcbrill committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    62a626b View commit details
    Browse the repository at this point in the history
  2. Merge branch 'master' into jbrill-msvc-channel

    Manually resolve conflicts:
    * SCons/Tool/MSCommon/MSVC/Util.py
    * SCons/Tool/MSCommon/MSVC/UtilTests.py
    jcbrill committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    2e0259c View commit details
    Browse the repository at this point in the history