Add preliminary support for linux-arm64

[amih90]

Add support for Debian11 (bulleye) on ARM64.

Issue describing the changes in this PR

part of #3112

Pull request checklist

• My changes do not require documentation changes
  • Otherwise: Documentation issue linked to PR
• My changes do not need to be backported to a previous version
  • Otherwise: Backport tracked by issue/PR #issue_or_pr
• I have added all required tests (Unit tests, E2E tests)

[EverybodyKurts]

@vrdmr @Francisco-Gamino @andystaples @amamounelsayed @ejizba @hossam-nasr @AnatoliB @khkh-ms

This PR was opened 2 months ago. Can somebody review it?

[khkh-ms]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[alecglen]

@khkh-ms Thanks for the review! FYI it appears the test failure is just a missing config in the test environment, and it's not clear to me why that'd be related to the current change.

[amih90]

/azp run

[azure-pipelines]

Commenter does not have sufficient privileges for PR 3584 in repo Azure/azure-functions-core-tools

[dirkesquire]

The test is failing with the single following error:
AZURE_MANAGEMENT_ACCESS_TOKEN is not defined in current environment

[dirkesquire]

I've determined that "AZURE_MANAGEMENT_ACCESS_TOKEN" is referred to in only two places:

azure-functions-core-tools/azure-pipelines.yml

Line 90 in 605b8b4

echo "##vso[task.setvariable variable=azure_management_access_token]$accessToken"

 - task: AzureCLI@2
    displayName: Login via Azure CLI to acquire access token
    inputs:
      azureSubscription: $(E2ETestServiceConnectionName)
      scriptType: ps
      scriptLocation: inlineScript
      inlineScript: |
        # acquire access token from Azure CLI and export it to AZURE_MANAGEMENT_ACCESS_TOKEN
        $accessToken = (az account get-access-token --query "accessToken" | % { $_.Trim('"') })
        echo "##vso[task.setvariable variable=azure_management_access_token]$accessToken"

azure-functions-core-tools/src/Azure.Functions.Cli/Common/Constants.cs

Line 53 in 605b8b4

public const string AzureManagementAccessToken = "AZURE_MANAGEMENT_ACCESS_TOKEN";

[dirkesquire]

The error is being thrown from this piece of code:

        protected static string AccessToken
        {
            get
            {
                if (_accessToken == null)
                {
                    _accessToken = Environment.GetEnvironmentVariable(Constants.AzureManagementAccessToken);
                    if (string.IsNullOrEmpty(_accessToken))
                    {
                        throw new Exception($"{Constants.AzureManagementAccessToken} is not defined in current environment");
                    }
                }
                return _accessToken;
            }
        }

at Azure.Functions.Cli.Tests.E2E.AzureResourceManagers.BaseAzureResourceManager.get_AccessToken()

Which seems to indicate that, exactly as expected and mentioned above, an environment variable is not being set for this test environment, and that that is the reason this entire PR is being held back, and not an issue with the code in the PR itself.

So what can we do next to push this forward? At the moment the community can not actually run Azure Functions in a dev container on a M1,M2 Mac. This is such a blocker. Should we be looking to try and build the solution locally, and patch it ourselves?

[dirkesquire]

I did manage to have some level of success with the fix.
I cloned azure-functions-core-tools locally, and applied the changes, and successfully built and published the solution so that I can now run func locally without errors. Hours spent on this because the Microsoft team does not have the time to spend on this PR.

The next problem that I am facing is that there is no worker for workers/python/3.11/LINUX/Arm64.
Please can someone give some advice on how I would arrange to create this worker.
(The worker LINUX/x86 does exist, but not the Arm64 one)

Thank you!

[alecglen]

@dirkesquire where is it missing from exactly? I've been trying to trace this through and it seems like the Arm64 version should exist: https://github.com/Azure/azure-functions-python-worker/blob/cd13c6aa1db0aba958f4aedded80254d69040899/python/prodV4/worker.config.json#L7

Requires more testing.

[joeuy]

Any updates?

[jhholm]

I managed to get this working in WSL2 with node.js with the premilinary changes in this pull request. I updated the changes mentioned here to the latest release. Would be really nice if this could be officially supported.

[dirkesquire]

I also managed to finally get this working for linux-arm64 inside a DevContainer on Apple Mac Silicon.
The python worker for linux-arm64 never gets populated and is completely missing. So it means you cannot run the func cli command inside a Devcontainer on any Apple Mac released after 2020 (i.e. with the M series Apple Silicon chip (not Intel)).
If I find some time I may try to submit a pull request.

@alecglen When you checkout azure-function-core-tools on the Release_4.0 branch and build and publish then you will notice that /path/to/publishfolder/workers/python/3.X/LINUX/Arm64 does not exist. Only the X64 version gets built for Linux currently.

[liliankasem]

Hi folks, ownership of this repo recently changed. I'll make this a priority for us to review & merge in our upcoming sprint.

path/to/publishfolder/workers/python/3.X/LINUX/Arm64 does not exist. Only the X64 version gets built for Linux currently.

I can also look into this as well

[JoJoAtkinson]

I’m incredibly excited to see this pull request completed! As someone who works with Azure Functions extensively, I believe this improvement will make a massive difference in my workflow and efficiency. I appreciate the effort that went into this—thank you for pushing this forward! @liliankasem

[liliankasem]

Hi folks, wanted to share an update on this.

Previously, python has been a blocker for getting arm64 out on other platforms, however I believe this is no longer the case and shouldn’t be blocking progress on this.

python/3.X/LINUX/Arm64 does not exist. Only the X64 version gets built for Linux currently.

The python team is looking into adding a arm64 build to the worker package. This will be a pre-requisite before we move forward.

As for next steps, we are thinking to take this into a feature branch to work on testing/validation. This will also allow us to release a preview version of Core Tools to gather feedback about potential issues.

Going forward, updates will be provided in the corresponding issue: #4279

[liliankasem]

/azp run

[azure-pipelines]

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

[liliankasem]

Merging and will address tests in my own PR.

[kurt-mueller-osumc]