Wrong detection of Visual Studio Express 2010 and 2012

[bdbaddog]

This issue was originally created at: 2012-12-05 14:55:15.
This issue was reported by: buildsmith.
buildsmith said at 2012-12-05 14:55:15

What is broken:

With the current code (2.2.0), when Express 2010 is installed, SCons wrongly believes that the non-Express version is installed as well. The very same happens for 2012 Express.

buildsmith said at 2012-12-05 15:04:38

How to fix it:

My way to fix it is to take profit of the a priori knowledge that Express and non-Express cannot coexist (the Microsoft installers take care of that, at least until next change of direction, which is hardly a surprise from them). There may be better ways to fix the detection.

Change 1: in get_installed_vcs() in file vc.py repolace the code

        try:
            if find_vc_pdir(ver):
                debug('found VC %s' % ver)
                installed_versions.append(ver)
            else:
                debug('find_vc_pdir return None for ver %s' % ver)
        except VisualCException, e:
            debug('did not find VC %s: caught exception %s' % (ver, str(e)))

with the code:

        try:
            if find_vc_pdir(ver):
                debug('found VC %s' % ver)
                ## Adrian: apriori knowldege injection here
                ## Express and Full cannot be installed side by side
                ## But our detection here makes it look like Express and Full are both installed 
                ## when in fact Express only is installed, for versions 2010 and 2012. So we filter out
                if ver.endswith("Exp") and len(installed_versions)>=1:
                    if ver.startswith(installed_versions[-1]):
                        ## oops, both Full and Express detected. Keep only Express
                        installed_versions[-1] = ver
                        debug('replacing VC Full with Express %s' % ver)
                    else:
                        installed_versions.append(ver)
                else:
                    installed_versions.append(ver)
            else:
                debug('find_vc_pdir return None for ver %s' % ver)
        except VisualCException, e:
            debug('did not find VC %s: caught exception %s' % (ver, str(e)))

buildsmith said at 2012-12-05 15:17:44

While we are at detecting things correctly, I suggest we fix also
the file sdk.py to add the Windows SDK in 2010.

Change 2: in the file sdk.py add this entry in the list:

# If you update this list, update the documentation in Tool/mssdk.xml.
SupportedSDKList = [
    WindowsSDK('7.0A',
               sanity_check_file=r'include\windows.h',
               include_subdir='include',
               lib_subdir={
                   'x86'       : ['lib'],
                   'x86_64'    : [r'lib\x64'],
                   'ia64'      : [r'lib\ia64'],
               },
               vc_setup_scripts = SDK70VCSetupScripts,
              ),

and then update the mssdk.xml as requested by the comment.

Remarks:

There is also a 8.0 SDK that comes with 2012. It is organized somewhat differently, so a simple extension like the one above may not do the job.

Fortunately, this sdk.py is not really useful anymore. Starting with the 2008 Express, the installers of Express have everything that it takes and the vcvars***.bat does find the relevant information for the SDKs as well. So sdk.py is now a hard requirement only for supporting 2005 Express (and the little-known 2003 compiler pack) and for people really using exotic extensions of the Windows API.

buildsmith said at 2012-12-06 01:32:48

fixing assignee. Sorry for the spam.

[mwichmann]

Again - very old issue, but have no clue if we correctly detect Express 2010/2012 - I'm assuming so? @jcbrill is there still anything to this?

[jcbrill]

@mwichmann SCons correctly detects MSVS Express 2010 and MSVS 2012 Express. The registry queries are correct for both. However, as stated above, SCons also detects that MSVS 2010 and MSVS 2012 are installed. The registry entries are also correct for the non-express versions and point to the same installed folders.

Because of the way amd64 targets are evaluated in the current SCons, using the msvc version without the "Exp" suffix now appears to work both with and without the "Exp" suffix. This was tested below. Basically the same msvc batch files are used (i.e., there are only 4 cache entries for the 8 builds). I can't be sure if this was always the case.

The SDK 7.0A as shown above is present in sdk.py (I claim no expertise with sdk.py).

As it happens, MSVS 2008 Professional was the last full version that I purchased. MSVS 2010 Express and MSVS 2012 Express are both installed on my development box. See the note below for targeting amd64 with MSVS 2010 Express.

The following SConstruct successfully compiles a simple "Hello, World" program for MSVS 2010 Express ('10.0Exp'), MSVS 2012 Express ('11.0Exp'), MSVS 2010 ('10.0') and MSVS 2011 ('11.0') targeting 'x86' and 'amd64' for each version:

import sys

env_list = []

def make_environment(**kwargs):
    global env_list
    build_n = len(env_list) + 1
    build = '_build{:03d}'.format(build_n)
    print('Build:', build, kwargs, file=sys.stderr)
    VariantDir(build, '.', duplicate=0)
    env=Environment(**kwargs)
    env.Program(build + '/hello', [build + '/hello.c'])
    env_list.append(env)
    return env

for d in [

    {'MSVC_VERSION': '10.0Exp', 'TARGET_ARCH': 'x86'   },
    {'MSVC_VERSION': '10.0Exp', 'TARGET_ARCH': 'amd64' },
    {'MSVC_VERSION': '11.0Exp', 'TARGET_ARCH': 'x86'   },
    {'MSVC_VERSION': '11.0Exp', 'TARGET_ARCH': 'amd64' },

    {'MSVC_VERSION': '10.0', 'TARGET_ARCH': 'x86'   },
    {'MSVC_VERSION': '10.0', 'TARGET_ARCH': 'amd64' },
    {'MSVC_VERSION': '11.0', 'TARGET_ARCH': 'x86'   },
    {'MSVC_VERSION': '11.0', 'TARGET_ARCH': 'amd64' },

]:
    make_environment(**d)

Output fragments with MSCommon debugging enabled:

Enabling scons cache ... 
Removing .scons_msvc_cache ... 
Removing build folders ... 
Running scons (15:35:17.06) ...
scons: Reading SConscript files ...
Build: _build001 {'MSVC_VERSION': '10.0Exp', 'TARGET_ARCH': 'x86'}
...
installed_vcs:['14.3', '14.2', '14.1', '14.1Exp', '14.0', '12.0', '11.0', '11.0Exp', '10.0', '10.0Exp', '8.0', '7.1', '7.0', '6.0']
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\vcvarsall.bat', args:x86
...
Build: _build002 {'MSVC_VERSION': '10.0Exp', 'TARGET_ARCH': 'amd64'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\vcvarsall.bat', args:amd64
...
Build: _build003 {'MSVC_VERSION': '11.0Exp', 'TARGET_ARCH': 'x86'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\VC\\vcvarsall.bat', args:x86
...
Build: _build004 {'MSVC_VERSION': '11.0Exp', 'TARGET_ARCH': 'amd64'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\VC\\vcvarsall.bat', args:x86_amd64
...
Build: _build005 {'MSVC_VERSION': '10.0', 'TARGET_ARCH': 'x86'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\vcvarsall.bat', args:x86
...
Build: _build006 {'MSVC_VERSION': '10.0', 'TARGET_ARCH': 'amd64'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\vcvarsall.bat', args:amd64
...
Build: _build007 {'MSVC_VERSION': '11.0', 'TARGET_ARCH': 'x86'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\VC\\vcvarsall.bat', args:x86
...
Build: _build008 {'MSVC_VERSION': '11.0', 'TARGET_ARCH': 'amd64'}
...
use_script 2 'C:\\Program Files (x86)\\Microsoft Visual Studio 11.0\\VC\\vcvarsall.bat', args:x86_amd64
...
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: _build001 _build002 _build003 _build004 _build005 _build006 _build007 _build008
cl /Fo_build008\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build008\hello.exe _build008\hello.obj
cl /Fo_build007\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build007\hello.exe _build007\hello.obj
cl /Fo_build006\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build006\hello.exe _build006\hello.obj
cl /Fo_build005\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build005\hello.exe _build005\hello.obj
cl /Fo_build004\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build004\hello.exe _build004\hello.obj
cl /Fo_build003\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build003\hello.exe _build003\hello.obj
cl /Fo_build002\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build002\hello.exe _build002\hello.obj
cl /Fo_build001\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build001\hello.exe _build001\hello.obj
scons: done building targets.
Scons finished (15:35:24.02)
Removing build folders ... 
Disabling scons cache ... 
Finished 

Note: MSVS 2010 Express does not include the 64-bit tools. The 64-bit tools must be installed via the Windows SDK 7.1. Once the SDK is installed there will not be a vcvars64.bat batch file in the amd folder. I use a vcvars64.bat that forwards the call to a batch that calls the SDK 7.1 SetEnv.cmd batch file. The indirection is necessary because (1) SetEnv.cmd expects delayed expansion to be enabled and (2) SetEnv.cmd expects environment variables that are not provided by scons.

[jcbrill]

Second explanation attempt...

The original post is not really whether MSVS 2012 Express and MSVS 2010 Express are detected correctly but rather when MSVS 2012 Express and MSVS 2010 Express are installed, MSVS 2012 and MSVS 2010 are detected as well.

The behavior identified in the original post still exists: when MSVS 2012 Express and MSVS 2010 Express are installed, the installed versions list from vc.py detection reports that 11.0, 11.0Exp, 10.0, and 10.0Exp are installed.

The original post rectifies this behavior by removing the "full" version from the installed list when the express version exists. Ultimately, the decision is whether this step is necessary or not.

This behavior is the result of the registry keys for the installers. The express versions and full versions generally cannot exist at the same time for 2012 and 2010. The express version and the full version are installed to the same default folder. The express registry keys and the "normal" registry keys both exist and point to the same folder which is why detection reports that both the express version and the full version are installed. The vc.py detection treats the same visual studio root folder as two different versions (express and full) as there is no check for the development environment binary.

From the perspective of vc.py (effectively the same as building from the command-line using the msvc tools) and the manner in which the target architectures are handled, there is not really a difference between the express and full versions as the development environment binary is not being invoked. As posted above, either seems to work fine.

The behavior still exists. It is probably easier to document the behavior than change the code. A better fix in the future would be to unify the vs and vc detection and only process visual studio roots once and classify the edition based on the installed binaries that are present.

PS: I have not checked the behavior in vs.py detection yet.

[mwichmann]

From the perspective of vc.py (effectively the same as building from the command-line using the msvc tools) and the manner in which the target architectures are handled, there is not really a difference between the express and full versions as the development environment binary is not being invoked. As posted above, either seems to work fine.

Okay, good to know.

The behavior still exists. It is probably easier to document the behavior than change the code. A better fix in the future would be to unify the vs and vc detection and only process visual studio roots once and classify the edition based on the installed binaries that are present.

Personally, and especially given the age of the products in question, I'd prefer the "document and move on" approach. Certainly open to suggestions on what to say and where to say it...

[bdbaddog]

Thinking from a users perspective:
If I request MSVS 2012 (not express), and MSVS 2012 Express is installed, SCons will beleive it is providing MSVS2012 (non express right)?

What is the harm in this mis-detection?

(I have a client who's stuck at MSVS 6, 2010 and a few other old versions, though they don't use SCons yet..)

[jcbrill]

What is the harm in this mis-detection?

As near as I can tell, there is no harm.

I'm not completely convinced it is a mis-detection from the command-line build tools perspective as the current vc.py implementation is somewhat flexible to the expected layout of the tools. However, an explicit user-defined HOST_ARCH might cause an issue depending on which msvc version (express vs full) was defined.

(I have a client who's stuck at MSVS 6, 2010 and a few other old versions, though they don't use SCons yet..)

I have legacy c-based dll's from past consulting projects that have msvc runtime dependencies for pretty much every version of MSVS since 6 (with the exception of 2002/2003). That is why I have all of the old versions installed.

[jcbrill]

IMHO, the exhibited behavior is a feature and not a bug. Perhaps this issue can be closed.

For msvc versions 9.0 to 14.0, if the only installed version of Visual Studio is an express edition, the non-express version (e.g., 9.0) is reported as being installed along with the express version (e.g., 9.0Exp).

Eliminating the so-called "mis-detection" may break existing builds and could make sconstruct files less portable. However, there are drawbacks based on feature availability. Also, not specifying a desired target architecture could lead to build failures (e.g., target architecture mis-match if linking external libraries). These drawbacks have existed since VS2008.

Modern express versions tend to support the same target architectures as the full editions with certain restrictions, if any, around advanced usage (e.g., store/UWP builds and/or sdk version specification).

The proposed branch (#4409) makes the non-express version (e.g., 14.1) an express version when the express version is the only edition installed for 14.1 and 8.0 to be consistent with the existing behavior. In simpler terms, an express version is effectively last in the list of detection for non-express versions.

The table below shows the existing and proposed detected versions in a VM with only express editions installed.

VCVer	MSVS	Main	#4409	Note
14.1			*	1
14.1Exp	*	*	*
14.0		*	*
14.0Exp	*		*	2
12.0		*	*
12.0Exp	*	*	*
11.0		*	*
11.0Exp	*	*	*
10.0		*	*
10.0Exp	*	*	*
9.0		*	*
9.0Exp	*	*	*
8.0			*	3
8.0Exp	*	*	*

Legend:

• VCVer: SCons MSVC_VERSION
• MSVS: Visual Studio installations
• Main: SCons master branch
• 4409: [WIP] MSVC: detection fixes and changes #4409
• Note: see below

Note:

1. PR will recognized Express version iff Express is the only edition installed.
2. 14.0Exp does not appear to be detected in main, corrected in PR.
3. PR will recognize Express version iff Express is the only edition installed.

Installed VCS (white-space added for visual comparison):

main: [        '14.1Exp', '14.0',            '12.0', '12.0Exp', '11.0', '11.0Exp', '10.0', '10.0Exp', '9.0', '9.0Exp',        '8.0Exp']
4409: ['14.1', '14.1Exp', '14.0', '14.0Exp', '12.0', '12.0Exp', '11.0', '11.0Exp', '10.0', '10.0Exp', '9.0', '9.0Exp', '8.0', '8.0Exp']

[mwichmann]

So "won't fix"? More than happy to close it out, it's ancient versions anyway.

[jcbrill]

Won't fix. Happy with the behavior.

I'm proposing that we make a newer version (e.g., 2017) work like the legacy behavior which can be discussed in the other PR.

[mwichmann]

ok, closing.