Exception "The path is not of a legal form" when loading .NET Standard 2.0 assembly

[ManfredLange]

Version 0.5 prerelease

Steps to reproduce:

1. Create a test project that targets .NET Standard 2.0
2. Reference a different assembly, the "system under test". The SUT is targeting .NET Framework 4.6.1
3. Build then load the assembly from the test project created in step 1
   Result: System.ArgumentException() in NUnit.Gui.Presenters.TestPropertiesPresenter.GetTestType(TestNode testNode) with message "The path is not of a legal form"

Call stack is:
System.ArgumentException: The path is not of a legal form.
at System.IO.Path.LegacyNormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths)
at System.IO.Path.GetFullPathInternal(String path)
at System.IO.FileInfo.Init(String fileName, Boolean checkHost)
at NUnit.Gui.Presenters.TestPropertiesPresenter.GetTestType(TestNode testNode) in C:\Dev\NUnit\nunit-gui\src\nunit-gui\Presenters\TestPropertiesPresenter.cs:line 212
at NUnit.Gui.Presenters.TestPropertiesPresenter.DisplayTestInfo(TestNode testNode) in C:\Dev\NUnit\nunit-gui\src\nunit-gui\Presenters\TestPropertiesPresenter.cs:line 99
at NUnit.Gui.Presenters.TestPropertiesPresenter.DisplaySelectedItem() in C:\Dev\NUnit\nunit-gui\src\nunit-gui\Presenters\TestPropertiesPresenter.cs:line 90
at NUnit.Gui.Presenters.TestPropertiesPresenter.b__4_4(TestItemEventArgs ea) in C:\Dev\NUnit\nunit-gui\src\nunit-gui\Presenters\TestPropertiesPresenter.cs:line 58
at NUnit.Gui.Model.TestItemEventHandler.Invoke(TestItemEventArgs args)
at NUnit.UiKit.Elements.TreeViewElement.<.ctor>b__3_0(Object s, TreeViewEventArgs e) in C:\Dev\NUnit\nunit-gui\src\nunit.uikit\Elements\TreeViewElement.cs:line 50
at System.Windows.Forms.TreeView.OnAfterSelect(TreeViewEventArgs e)
at System.Windows.Forms.TreeView.TvnSelected(NMTREEVIEW* nmtv)
at System.Windows.Forms.TreeView.WmNotify(Message& m)
at System.Windows.Forms.TreeView.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[immeraufdemhund]

I am able to confirm this on my machine. I hope that i can dig into it tonight after work

[mikkelbu]

It is probably something similar to TestCentric/testcentric-gui#184. AFAICR I actually had a patch - somewhere - containing some better error handling when we have fullname="" (or perhaps I just returned testNode.Type if fullname=="" instead of trying to create a FileInfo)
https://github.com/NUnitSoftware/nunit-gui/blob/221c23c31f14e3db82be9419ac1b53e894bc565d/src/nunit-gui/Presenters/TestPropertiesPresenter.cs#L211).

[immeraufdemhund]

indeed this is the problem @mikkelbu . the TestNode's XML is

<test-suite type='Assembly' id='0' name='' fullname='' runstate='NotRunnable' testcasecount='1' result='Failed' label='Error' start-time='2017-09-08 00:36:54Z' end-time='2017-09-08 00:36:54Z' duration='0.001' total='1' passed='0' failed='1' inconclusive='0' skipped='0' asserts='0'>
    <failure>
        <message><![CDATA[Could not load file or assembly 'nunit.framework' or one of its dependencies. The system cannot find the file specified.]]></message>
        <stack-trace><![CDATA[
Server stack trace: 
   at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
   at System.Activator.CreateInstance(String assemblyString, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Engine.Drivers.NUnit3FrameworkDriver.Load(String testAssemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.Explore(TestFilter filter)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at NUnit.Engine.ITestEngineRunner.Explore(TestFilter filter)
   at NUnit.Engine.Runners.ProcessRunner.Explore(TestFilter filter)]]></stack-trace>
    </failure>
</test-suite>

So what if we check for a failure on the testNode.Xml.FirstChild.Name is failure then show the message??? seems a little weird. That or I suppose change testNode to show a error if it has one.

[ManfredLange]

There is a possibility that a netstandard2.0 project referencing a net461 project is not designed to work.

Having said that, a netstandard2.0 project can reference net461 nuget packages, though. At best there will be a nuget warning that the net461 nuget package does not support netstandard2.0 and therefore the whole thing may not run in the target environment.

I don't know whether this makes a difference for this particular issue. It shouldn't impact the GUI, though, I think, as it should not make a difference what and how the assembly containing the tests references a difference package, project or assembly.

[CharliePoole]

Test assemblies have to target a platform, not .NET standard.

[ManfredLange]

Bad choice of words on my end.

Looking at a new format csproj file, there is a property named TargetFramework that you can set to different values, e.g. net461 or netstandard2.0. That is what I meant with by using the term "netstandard2.0 project", "netstandard2.0 nuget packages". Replacing TargetFramework with TargetFrameworks allows targeting multiple frameworkes, e.g. net461 and netstandard2.0.

Yes, ultimately, even if the project targets one of those, the assembly can then be executed only if the respective target framework is available on the platform on which it runs. So, in case of net461 it may only be runnable on a Windows box. In the case of targeting netstandard2.0, the resulting is only runnable if a .NET Framework is available that implements .NET Standard 2.0, e.g. .NET Core 2.0 or .NET Framework 4.6.1. And even if it targets netstandard2.0 it may still fail in some cases. For example some of the API's may not be implemented (e.g. AppDomain comes to mind).

What I tried to convey is this: given a project that targets framework netstandard2.0. It seems to be fine to reference a nuget package that targets net4.6.1 only. However, it becomes problematic if the same project has a dependency on a different project (rather than package) targeting net4.6.1.

Therefore in our 50+ projects solutions we have structured the dependencies in such a way that projects targeting net461 can have a dependency on a project targeting netstandard2.0. We don't have dependencies the other way round between projects. We have projects targeting netstandard2.0, though, which have dependencies on nuget packages that target net461 (or other full .NET Framework versions). This approach allows us migrating the solution incrementally towards .NET Standard/Core 2.0.

[ef32]

I got this issue when i downloaded nunit-gui 3, I cant seem to run any of my tests , some help will be appreciated

[CharliePoole]

@ef2 As we frequently point out 😄 this is an experimental, pre-release project, which we don't recommend anyone use for production work. A fix was put in to avoid the gui crash, but not to make the net standard tests actually run.

Because the current design relies on the nunit test engine, which doesn't support .NET Standard, it's unlikely we will have this gui running .NET Standard tests any time soon. We will be doing a review to decide whether to continue on this path or take a new approach.

I'm sorry that there isn't a better answer. 😢