Merge pull request #8705 from drewnoakes/build-acceleration

[17.5p2] Accelerate builds in VS by copying files directly

Directory.Build.props

@@ -16,6 +16,7 @@
     <BaseIntermediateOutputPath Condition="'$(BaseIntermediateOutputPath)' == ''">$([System.IO.Path]::GetFullPath('$(ArtifactsObjDir)$(MSBuildProjectName)\'))</BaseIntermediateOutputPath>
     <IntermediateOutputPath>$(BaseIntermediateOutputPath)</IntermediateOutputPath>
     <UseCommonOutputDirectory Condition="'$(UseCommonOutputDirectory)' == ''">true</UseCommonOutputDirectory>
+    <AccelerateBuildsInVisualStudio>true</AccelerateBuildsInVisualStudio>
   </PropertyGroup>
 
   <!-- Configuration -->

docs/build-acceleration.md

@@ -0,0 +1,74 @@
+# Build acceleration
+
+Build acceleration is a feature of Visual Studio that reduces the time required to build projects.
+
+The feature was added in 17.5 and is currently opt-in. It applies to SDK-style .NET projects only. It is simple to try, and in most cases will improve build times. Larger solutions will see greater gains.
+
+This document will outline what the feature does, how to enable it, and when it might not be suitable.
+
+## Background
+
+Visual Studio uses MSBuild to build .NET projects. There is some overhead associated with calling MSBuild to build each project, so Visual Studio uses a "fast up-to-date check" (FUTDC) to avoid calling MSBuild unless needed. This FUTDC can quickly determine if anything has changed in the project that would cause a build to be required. For more information, see [Up-to-date Check](up-to-date-check.md).
+
+In several cases, the FUTDC identifies that no compilation is required, yet identifies some files need to be copied to the output directory, either from the current project or from a referenced one. Historically in this scenario, the FUTDC would call MSBuild to build the project, even though no compilation was required. This was done to ensure that the files were copied to the output directory.
+
+With the build acceleration feature, Visual Studio will perform these files copies directly rather than calling MSBuild to do them.
+
+## Example
+
+Build acceleration is particularly impactful when changes need to be copied around several times during build.
+
+```mermaid
+graph LR
+    A[Unit Test] --> B[Library 1]
+    B --> C[Library 2]
+    C --> D[Library 3]
+```
+
+Consider this example, where a unit test project references a project that in turn references another, and so on.
+
+Making a change in _Library 3_ and running the unit test would previously have caused four calls to MSBuild.
+
+With build acceleration enabled MSBuild is called only once, after which VS copies the output of _Library 3_ to all referencing projects
+
+## Configuration
+
+Build acceleration is currently opt-in.
+
+To enable it in your solution, add or edit a top-level [`Directory.Build.props`](https://learn.microsoft.com/visualstudio/msbuild/customize-your-build) file to include:
+
+```xml
+<Project>
+  <PropertyGroup>
+    <AccelerateBuildsInVisualStudio>true</AccelerateBuildsInVisualStudio>
+  </PropertyGroup>
+</Project>
+```
+
+You may disable build acceleration for specific projects in your solution by redefining the `AccelerateBuildsInVisualStudio` property as `false` in those projects.
+
+## Debugging
+
+Build acceleration runs with the FUTDC, and outputs details of its operation in the build log. To enable this logging:
+
+> Tools | Options | Projects and Solutions | .NET Core
+
+![Projects and Solutions, .NET Core options](repo/images/options.png)
+
+Setting _Logging Level_ to a value other than `None` results in messages prefixed with `FastUpToDate:` in Visual Studio's build output.
+
+- `None` disables log output.
+- `Minimal` produces a single message per out-of-date project.
+- `Info` and `Verbose` provide increasingly detailed information about the inner workings of the check, which are useful for debugging.
+
+## Limitations
+
+MSBuild is very configurable, and there are many ways to configure a project that will prevent build acceleration from working correctly. For example, if a project's build defines post-compile steps that are important to the correct functioning of your project, then build acceleration will not correctly reproduce those steps when it bypasses MSBuild.
+
+Note that NuGet packages can modify a project's build in non-obvious ways that may have undesirable interactions with build acceleration.
+
+We recommend enabling build acceleration for all projects in the solution, as described above, then monitoring for any unexpected behavior. You can use the log output to verify whether build acceleration is the culprit. If so, disable it for that project.
+
+## Giving feedback
+
+If you encounter an issue with build acceleration, please [file an issue](https://github.com/dotnet/project-system/issues/new/choose) and we will investigate whether it's something that can be addressed.

setup/Microsoft.VisualStudio.ProjectSystem.Managed.CommonFiles/Microsoft.VisualStudio.ProjectSystem.Managed.CommonFiles.csproj

@@ -8,6 +8,11 @@
 
   <Import Project="..\..\eng\imports\VisualStudio.props" />
 
+  <PropertyGroup>
+    <!-- This VSSDK project has custom build steps and should not be accelerated. -->
+    <AccelerateBuildsInVisualStudio>false</AccelerateBuildsInVisualStudio>
+  </PropertyGroup>
+
   <ItemGroup>
     <!-- Depend on projects producing XAML rules included in this Willow package -->
     <ProjectReference Include="..\..\src\Microsoft.VisualStudio.ProjectSystem.Managed\Microsoft.VisualStudio.ProjectSystem.Managed.csproj" />

setup/ProjectSystemSetup/ProjectSystemSetup.csproj

@@ -15,6 +15,9 @@
     <ExtensionInstallationFolder>Microsoft\ManagedProjectSystem</ExtensionInstallationFolder>
     <TargetVsixContainerName>$(AssemblyName).vsix</TargetVsixContainerName>
     <TargetVsixContainer>$(VisualStudioSetupInsertionPath)$(TargetVsixContainerName)</TargetVsixContainer>
+
+    <!-- This VSSDK project has custom build steps and should not be accelerated. -->
+    <AccelerateBuildsInVisualStudio>false</AccelerateBuildsInVisualStudio>
   </PropertyGroup>
 
   <!-- Local properties -->

setup/VisualStudioEditorsSetup/VisualStudioEditorsSetup.csproj

@@ -13,6 +13,9 @@
     <ExtensionInstallationFolder>Microsoft\VisualStudio\Editors</ExtensionInstallationFolder>
     <TargetVsixContainerName>$(AssemblyName).vsix</TargetVsixContainerName>
     <TargetVsixContainer>$(VisualStudioSetupInsertionPath)$(TargetVsixContainerName)</TargetVsixContainer>
+
+    <!-- This VSSDK project has custom build steps and should not be accelerated. -->
+    <AccelerateBuildsInVisualStudio>false</AccelerateBuildsInVisualStudio>
   </PropertyGroup>
 
   <ItemGroup>

src/Microsoft.VisualStudio.ProjectSystem.Managed.VS/ProjectSystem/VS/UpToDate/UpToDateCheckBuildEventNotifier.cs

@@ -10,14 +10,15 @@ namespace Microsoft.VisualStudio.ProjectSystem.VS.UpToDate
 {
     /// <summary>
     /// Listens for build events and notifies the fast up-to-date check of them
-    /// via <see cref="IBuildUpToDateCheckProviderInternal"/>.
+    /// via <see cref="IProjectBuildEventListener"/>.
     /// </summary>
     [Export(ExportContractNames.Scopes.UnconfiguredProject, typeof(IProjectDynamicLoadComponent))]
     [AppliesTo(BuildUpToDateCheck.AppliesToExpression)]
     internal sealed class UpToDateCheckBuildEventNotifier : OnceInitializedOnceDisposedAsync, IVsUpdateSolutionEvents2, IProjectDynamicLoadComponent
     {
         private readonly IProjectService _projectService;
         private readonly ISolutionBuildManager _solutionBuildManager;
+        private readonly ISolutionBuildEventListener _solutionBuildEventListener;
         private readonly IProjectThreadingService _threadingService;
         private readonly IProjectFaultHandlerService _faultHandlerService;
         private IAsyncDisposable? _solutionBuildEventsSubscription;
@@ -28,13 +29,15 @@ public UpToDateCheckBuildEventNotifier(
             IProjectService projectService,
             IProjectThreadingService threadingService,
             IProjectFaultHandlerService faultHandlerService,
-            ISolutionBuildManager solutionBuildManager)
+            ISolutionBuildManager solutionBuildManager,
+            ISolutionBuildEventListener solutionBuildEventListener)
             : base(new(joinableTaskContext))
         {
             _projectService = projectService;
             _threadingService = threadingService;
             _faultHandlerService = faultHandlerService;
             _solutionBuildManager = solutionBuildManager;
+            _solutionBuildEventListener = solutionBuildEventListener;
         }
 
         public Task LoadAsync() => InitializeAsync();
@@ -65,13 +68,15 @@ int IVsUpdateSolutionEvents2.UpdateProjectCfg_Begin(IVsHierarchy pHierProj, IVsC
         {
             if (IsBuild(dwAction, out _))
             {
-                IEnumerable<IBuildUpToDateCheckProviderInternal>? providers = FindActiveConfiguredProviders(pHierProj, out _);
+                IEnumerable<IProjectBuildEventListener>? listeners = FindActiveConfiguredProviders(pHierProj, out _);
 
-                if (providers is not null)
+                if (listeners is not null)
                 {
-                    foreach (IBuildUpToDateCheckProviderInternal provider in providers)
+                    var buildStartedTimeUtc = DateTime.UtcNow;
+
+                    foreach (IProjectBuildEventListener listener in listeners)
                     {
-                        provider.NotifyBuildStarting(DateTime.UtcNow);
+                        listener.NotifyBuildStarting(buildStartedTimeUtc);
                     }
                 }
             }
@@ -86,18 +91,18 @@ int IVsUpdateSolutionEvents2.UpdateProjectCfg_Done(IVsHierarchy pHierProj, IVsCf
         {
             if (fCancel == 0 && IsBuild(dwAction, out bool isRebuild))
             {
-                IEnumerable<IBuildUpToDateCheckProviderInternal>? providers = FindActiveConfiguredProviders(pHierProj, out UnconfiguredProject? unconfiguredProject);
+                IEnumerable<IProjectBuildEventListener>? listeners = FindActiveConfiguredProviders(pHierProj, out UnconfiguredProject? unconfiguredProject);
 
-                if (providers is not null)
+                if (listeners is not null)
                 {
                     JoinableTask task = _threadingService.JoinableTaskFactory.RunAsync(async () =>
                     {
                         // Do this work off the main thread
                         await TaskScheduler.Default;
 
-                        foreach (IBuildUpToDateCheckProviderInternal provider in providers)
+                        foreach (IProjectBuildEventListener listener in listeners)
                         {
-                            await provider.NotifyBuildCompletedAsync(wasSuccessful: fSuccess != 0, isRebuild);
+                            await listener.NotifyBuildCompletedAsync(wasSuccessful: fSuccess != 0, isRebuild);
                         }
                     });
 
@@ -123,7 +128,7 @@ private static bool IsBuild(uint options, out bool isRebuild)
             return (operation & anyBuildFlags) == anyBuildFlags;
         }
 
-        private IEnumerable<IBuildUpToDateCheckProviderInternal>? FindActiveConfiguredProviders(IVsHierarchy vsHierarchy, out UnconfiguredProject? unconfiguredProject)
+        private IEnumerable<IProjectBuildEventListener>? FindActiveConfiguredProviders(IVsHierarchy vsHierarchy, out UnconfiguredProject? unconfiguredProject)
         {
             unconfiguredProject = _projectService.GetUnconfiguredProject(vsHierarchy, appliesToExpression: BuildUpToDateCheck.AppliesToExpression);
 
@@ -135,20 +140,37 @@ private static bool IsBuild(uint options, out bool isRebuild)
 
                 if (configuredProject is not null)
                 {
-                    return configuredProject.Services.ExportProvider.GetExportedValues<IBuildUpToDateCheckProviderInternal>();
+                    return configuredProject.Services.ExportProvider.GetExportedValues<IProjectBuildEventListener>();
                 }
             }
 
             return null;
         }
 
+        int IVsUpdateSolutionEvents.UpdateSolution_StartUpdate(ref int pfCancelUpdate)
+        {
+            _solutionBuildEventListener.NotifySolutionBuildStarting(DateTime.UtcNow);
+
+            return HResult.OK;
+        }
+        int IVsUpdateSolutionEvents.UpdateSolution_Done(int fSucceeded, int fModified, int fCancelCommand)
+        {
+            _solutionBuildEventListener.NotifySolutionBuildCompleted();
+
+            return HResult.OK;
+        }
+
+        int IVsUpdateSolutionEvents.UpdateSolution_Cancel()
+        {
+            _solutionBuildEventListener.NotifySolutionBuildCompleted();
+
+            return HResult.OK;
+        }
+
         #region IVsUpdateSolutionEvents stubs
 
-        int IVsUpdateSolutionEvents.UpdateSolution_StartUpdate(ref int pfCancelUpdate) => HResult.OK;
-        int IVsUpdateSolutionEvents.UpdateSolution_Begin(ref int pfCancelUpdate) => HResult.OK;
-        int IVsUpdateSolutionEvents.UpdateSolution_Done(int fSucceeded, int fModified, int fCancelCommand) => HResult.OK;
-        int IVsUpdateSolutionEvents.UpdateSolution_Cancel() => HResult.OK;
         int IVsUpdateSolutionEvents.OnActiveProjectCfgChange(IVsHierarchy pIVsHierarchy) => HResult.OK;
+        int IVsUpdateSolutionEvents.UpdateSolution_Begin(ref int pfCancelUpdate) => HResult.OK;
 
         int IVsUpdateSolutionEvents2.OnActiveProjectCfgChange(IVsHierarchy pIVsHierarchy) => HResult.OK;
         int IVsUpdateSolutionEvents2.UpdateSolution_Begin(ref int pfCancelUpdate) => HResult.OK;

src/Microsoft.VisualStudio.ProjectSystem.Managed/IO/IFileSystem.cs

@@ -50,6 +50,8 @@ internal interface IFileSystem
 
         bool TryGetFileSizeBytes(string path, out long result);
 
+        (long SizeBytes, DateTime WriteTimeUtc)? GetFileSizeAndWriteTimeUtc(string path);
+
         bool DirectoryExists(string path);
         void CreateDirectory(string path);
         string GetFullPath(string path);

src/Microsoft.VisualStudio.ProjectSystem.Managed/IO/Win32FileSystem.cs

@@ -115,6 +115,18 @@ public bool TryGetFileSizeBytes(string path, out long result)
             return false;
         }
 
+        public (long SizeBytes, DateTime WriteTimeUtc)? GetFileSizeAndWriteTimeUtc(string path)
+        {
+            var info = new FileInfo(path);
+
+            if (info.Exists)
+            {
+                return (info.Length, info.LastWriteTimeUtc);
+            }
+
+            return null;
+        }
+
         public bool DirectoryExists(string dirPath)
         {
             return Directory.Exists(dirPath);

src/Microsoft.VisualStudio.ProjectSystem.Managed/Microsoft.VisualStudio.ProjectSystem.Managed.csproj

@@ -577,6 +577,18 @@
       <RuleInjection>None</RuleInjection>
     </XamlPropertyRule>
 
+    <Compile Update="ProjectSystem\Rules\CopyToOutputDirectoryItem.cs">
+      <DependentUpon>CopyToOutputDirectoryItem.xaml</DependentUpon>
+    </Compile>
+    <XamlPropertyRule Include="ProjectSystem\Rules\CopyToOutputDirectoryItem.xaml">
+      <CopyToOutputDirectory>Never</CopyToOutputDirectory>
+      <XlfInput>false</XlfInput>
+      <Generator>MSBuild:GenerateRuleSourceFromXaml</Generator>
+      <SubType>Designer</SubType>
+      <DataAccess>None</DataAccess>
+      <RuleInjection>None</RuleInjection>
+    </XamlPropertyRule>
+
     <Compile Update="ProjectSystem\Rules\WindowsFormsConfiguration.cs">
       <DependentUpon>WindowsFormsConfiguration.xaml</DependentUpon>
     </Compile>

src/Microsoft.VisualStudio.ProjectSystem.Managed/ProjectSystem/DesignTimeTargets/Microsoft.Managed.DesignTime.targets

@@ -472,4 +472,64 @@
     </ItemGroup>
   </Target>
 
+  <!-- This target collects all the items this project contributes to output directories of referencing projects.
+       Ideally, it excludes any transitive dependencies.
+  -->
+  <Target
+    Name="CollectCopyToOutputDirectoryItemDesignTime"
+    DependsOnTargets="PrepareResourceNames;ComputeIntermediateSatelliteAssemblies;ResolveAssemblyReferences;CompileDesignTime;_GetCopyToOutputDirectoryItemsFromThisProject"
+    Returns="@(_CollectedCopyToOutputDirectoryItem)">
+    <ItemGroup>
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(_ThisProjectItemsToCopyToOutputDirectory)" />
+
+      <!-- Output assembly -->
+
+      <_CollectedCopyToOutputDirectoryItem Include="$(TargetPath)">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>$([System.IO.Path]::GetFileName('$(TargetPath)'))</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+      <!-- Debug symbols -->
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(_DebugSymbolsOutputPath->'%(FullPath)')" Condition="'$(_DebugSymbolsProduced)' == 'true' and '$(SkipCopyingSymbolsToOutputDirectory)' != 'true' and '$(CopyOutputSymbolsToOutputDirectory)' != 'false'">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>$([System.IO.Path]::GetFileName('%(Identity)'))</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+      <!-- ReferenceCopyLocalPaths
+
+           These come from two different sources, hence projecting twice here.
+           We filter out items that come from NuGet packages as there can be very many of them, and we exclude
+           them in the project system anyway, so it's better to drop them here (at the source). These items are
+           expected to be copied during an actual build, and need updating during subsequent incremental builds.
+      -->
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(ReferenceCopyLocalPaths->'%(Identity)')" Condition="'%(ReferenceCopyLocalPaths.ReferenceSourceTarget)' != 'ProjectReference' AND '%(ReferenceCopyLocalPaths.DestinationSubPath)' != ''">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>%(ReferenceCopyLocalPaths.DestinationSubPath)</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(ReferenceCopyLocalPaths->'%(Identity)')" Condition="'%(ReferenceCopyLocalPaths.ReferenceSourceTarget)' != 'ProjectReference' AND '%(ReferenceCopyLocalPaths.DestinationSubPath)' == ''">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>$([System.IO.Path]::GetFileName('%(Identity)'))</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+      <!-- Documentation file -->
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(FinalDocFile->'%(FullPath)')" Condition="'$(_DocumentationFileProduced)' == 'true'">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>$([System.IO.Path]::GetFileName('%(Identity)'))</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+      <!-- Satellite assemblies -->
+
+      <_CollectedCopyToOutputDirectoryItem Include="@(IntermediateSatelliteAssembliesWithTargetPath->'$(TargetDir)%(Culture)\$(TargetName).resources.dll')" Condition="'@(IntermediateSatelliteAssembliesWithTargetPath)' != ''">
+        <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
+        <TargetPath>%(IntermediateSatelliteAssembliesWithTargetPath.Culture)\$(TargetName).resources.dll</TargetPath>
+      </_CollectedCopyToOutputDirectoryItem>
+
+    </ItemGroup>
+  </Target>
+
 </Project>