Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prejit support on FunctionsNetHost #2711

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open

Prejit support on FunctionsNetHost #2711

wants to merge 24 commits into from

Conversation

kshyju
Copy link
Member

@kshyju kshyju commented Sep 13, 2024

Fixes #2637

This PR adds support for pre-jitting framework methods during placeholder mode. Currently this is enabled only for net8 onwards. This behavior can be explicitly disabled by setting AZURE_FUNCTIONS_FUNCTIONSNETHOST_DISABLE_PREJIT environment variable value to 0.

See internal wiki page for more context.

I will have a separate follow up PR for fixing the build pipeline & nuget packaging needed for this change. Also included the build pipeline changes to include the new artifacts in the final package.

Did a minor version bump.

Pull request checklist

  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • I have added all required tests (Unit tests, E2E tests)

@kshyju
Copy link
Member Author

kshyju commented Sep 13, 2024

@steveisok @ivdiazsa FYI.

@kshyju
Copy link
Member Author

kshyju commented Sep 13, 2024

@khkh-ms FYI. I used a previously generated trace for this PR. Once your changes are done, we could replace this file with that. Let's merge both traces to single file (like what we do for host)

@kshyju
Copy link
Member Author

kshyju commented Sep 18, 2024

@steveisok @ivdiazsa Appreciate if you could do review the changes.

Copy link
Member

@liliankasem liliankasem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you added unit tests where possible?

host/src/FunctionsNetHost.Shared/EnvironmentVariables.cs Outdated Show resolved Hide resolved
host/src/FunctionsNetHost/PreJit/PreJitManager.cs Outdated Show resolved Hide resolved
host/src/FunctionsNetHost/PreJit/PreJitManager.cs Outdated Show resolved Hide resolved
host/src/FunctionsNetHost/PreJit/PreJitManager.cs Outdated Show resolved Hide resolved
host/src/PlaceholderApp/JitTrace/JitTraceRuntime.cs Outdated Show resolved Hide resolved
host/src/PlaceholderApp/StartupHook.cs Outdated Show resolved Hide resolved
host/src/global.json Outdated Show resolved Hide resolved
host/src/PlaceholderApp/StartupHook.cs Outdated Show resolved Hide resolved
@ivdiazsa
Copy link

I don't have further comments other than addressing the ones that have been pointed out so far.

@kshyju
Copy link
Member Author

kshyju commented Sep 19, 2024

Can you add unit tests where possible?

We don't have any tests as of today as what we want to validate is the native output. There has been discussion about adding an E2E test which simulates the linux scenarios as part of build pipeline (potentially a nightly build). Never got a chance to create a work item until now. #2724

I think that should be the priority. We can certainly revisit as we have more cycles to work on this.

@steveisok
Copy link

Looks good to me! Glad you were able to work this in 👍

Copy link

@safihamid safihamid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@@ -25,5 +25,5 @@ jobs:
publishWebProjects: false
zipAfterPublish: false
modifyOutputPath: false
arguments: -c Release -o $(Build.ArtifactStagingDirectory)/_preLaunchAppPackages/${{ version }} -f ${{ version }} -p:UseAppHost=false
arguments: -c Release -o $(Build.ArtifactStagingDirectory)/_preLaunchAppPackages/${{ replace(version, 'net', '') }} -f ${{ version }} -p:UseAppHost=false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixing an existing error in output path value, which was introduced during the pipeline migration.

@kshyju kshyju requested a review from fabiocav September 23, 2024 02:06
@@ -0,0 +1,331 @@
[S.P.CoreLib]System.Array.Empty<Candidate>()~System.Array,System.Private.CoreLib~0~1:Microsoft.AspNetCore.Routing.Matching.Candidate,Microsoft.AspNetCore.Routing~Empty
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eliaslopezgt please replace coldstart.jittrace and linux.coldstart.jittrace files.

@kshyju
Copy link
Member Author

kshyju commented Sep 27, 2024

E2E test results

Windows

Validated E2E in private stamp. Everything works as expected.

Linux

Specialization is working as expected, but the http trigger invocation fails with below error.

Function.HelloHttp[3]
2024-09-27 06:50:31 Executed 'Functions.HelloHttp' (Failed, Id=98960abe-dec0-4665-bd88-d0e1a93d8092, Duration=5ms)
2024-09-27 06:50:31 Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.HelloHttp
2024-09-27 06:50:31 ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException: Result: Failure
2024-09-27 06:50:31 Exception: System.IO.FileNotFoundException: Could not load file or assembly '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'. The system cannot find the file specified.
2024-09-27 06:50:31
2024-09-27 06:50:31 File name: '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'
2024-09-27 06:50:31 at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 Stack: at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 101
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionGenerator.cs:line 225 2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync(Object instance, Object[] arguments) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 581

The GrpcFunctionDefinition code relies on FUNCTIONS_APPLICATION_DIRECTORY and FUNCTIONS_WORKER_DIRECTORY (as fallback) environment variable values to build the path used to load the assembly. Need to investigate further why this value is coming as incorrect.

string? scriptRoot = Environment.GetEnvironmentVariable(FunctionsApplicationDirectoryKey) ?? Environment.GetEnvironmentVariable(FunctionsWorkerDirectoryKey);
if (string.IsNullOrWhiteSpace(scriptRoot))
{
throw new InvalidOperationException($"The '{FunctionsApplicationDirectoryKey}' environment variable value is not defined. This is a required environment variable that is automatically set by the Azure Functions runtime.");
}
if (string.IsNullOrWhiteSpace(loadRequest.Metadata.ScriptFile))
{
throw new InvalidOperationException($"Metadata for function '{loadRequest.Metadata.Name} ({loadRequest.Metadata.FunctionId})' does not specify a 'ScriptFile'.");
}
string scriptFile = Path.Combine(scriptRoot, loadRequest.Metadata.ScriptFile);
PathToAssembly = Path.GetFullPath(scriptFile);

Log("Waiting for cold start request.");

// When specialization request arrives, FNH will connect to this named server stream and send the assembly path.
using (var pipeServer = new NamedPipeServerStream(Constants.NetHostWaitHandleName, PipeDirection.In))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we instead make an interop call from the native host following the pattern used against the worker?

src/DotNetWorker.Core/StartupHook.cs Outdated Show resolved Hide resolved
<TargetFrameworks>net8.0;net9.0</TargetFrameworks>
<DebugSymbols>False</DebugSymbols>
<DebugType>None</DebugType>
<TieredCompilation>false</TieredCompilation>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set all appropriate assembly properties here.

This assembly will be injected and visible in the default context. It should be signed, properly versioned and named

var taEx = new System.Threading.ThreadInterruptedException();

// Same with ASP.NET Core types.
var provider = new Microsoft.Extensions.Configuration.EnvironmentVariables.EnvironmentVariablesConfigurationProvider();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are this part of our safe list of dependencies that won't ship out-of-band?

Assembly entryAssembly = Assembly.GetEntryAssembly();
string version = System.Diagnostics.FileVersionInfo.GetVersionInfo(entryAssembly.Location).FileVersion;

var jsonPath = Path.Combine(Path.GetDirectoryName(entryAssembly.Location), Path.GetFileNameWithoutExtension(entryAssembly.Location) + ".deps.json");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear whether this is simply trying to exercise these paths or if there's something missing here. Can you comment?

<TieredCompilation>false</TieredCompilation>
</PropertyGroup>
<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not part of our safe list of dependencies to bring in. At least in this iteration, we don't want to have this present.

@kshyju
Copy link
Member Author

kshyju commented Nov 1, 2024

This pull request has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.

Domo.

outputs:
- output: pipelineArtifact
displayName: Publish Pre-jit placeholder app artifacts
path: $(Build.ArtifactStagingDirectory)/_preJitAppPackages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider making this a variable

steps:
- template: /eng/ci/templates/steps/install-dotnet.yml@self

- ${{ each version in split(variables.dotnetVersions, ',') }}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit: use framework terminology instead of version.

path: $(Build.SourcesDirectory)/host/dist/portable
path: $(Build.SourcesDirectory)/host/dist/portable/prelaunchapps

- task: DownloadPipelineArtifact@2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we supposed to use DownloadPipelineArtifact? 1ES has its own syntax for artifact inputs:

https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/1es-pipeline-templates/features/inputs/pipeline-artifact

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a separate solution file, can we not use an existing one?

@kshyju
Copy link
Member Author

kshyju commented Nov 15, 2024

E2E test results

Windows

Validated E2E in private stamp. Everything works as expected.

Linux

Specialization is working as expected, but the http trigger invocation fails with below error.

Function.HelloHttp[3]
2024-09-27 06:50:31 Executed 'Functions.HelloHttp' (Failed, Id=98960abe-dec0-4665-bd88-d0e1a93d8092, Duration=5ms)
2024-09-27 06:50:31 Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.HelloHttp
2024-09-27 06:50:31 ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException: Result: Failure
2024-09-27 06:50:31 Exception: System.IO.FileNotFoundException: Could not load file or assembly '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'. The system cannot find the file specified.
2024-09-27 06:50:31
2024-09-27 06:50:31 File name: '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'
2024-09-27 06:50:31 at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 Stack: at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 101
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionGenerator.cs:line 225 2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync(Object instance, Object[] arguments) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 581

The GrpcFunctionDefinition code relies on FUNCTIONS_APPLICATION_DIRECTORY and FUNCTIONS_WORKER_DIRECTORY (as fallback) environment variable values to build the path used to load the assembly. Need to investigate further why this value is coming as incorrect.

string? scriptRoot = Environment.GetEnvironmentVariable(FunctionsApplicationDirectoryKey) ?? Environment.GetEnvironmentVariable(FunctionsWorkerDirectoryKey);
if (string.IsNullOrWhiteSpace(scriptRoot))
{
throw new InvalidOperationException($"The '{FunctionsApplicationDirectoryKey}' environment variable value is not defined. This is a required environment variable that is automatically set by the Azure Functions runtime.");
}
if (string.IsNullOrWhiteSpace(loadRequest.Metadata.ScriptFile))
{
throw new InvalidOperationException($"Metadata for function '{loadRequest.Metadata.Name} ({loadRequest.Metadata.FunctionId})' does not specify a 'ScriptFile'.");
}
string scriptFile = Path.Combine(scriptRoot, loadRequest.Metadata.ScriptFile);
PathToAssembly = Path.GetFullPath(scriptFile);

On Unix-like systems, in-process environment modifications made by native libraries aren't seen by managed callers. - This is what is causing the failure.

@kshyju kshyju force-pushed the shkr/2637_fnh_prejit branch from 306afe0 to 4310a3d Compare November 15, 2024 23:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prejit support on FunctionsNetHost
7 participants