IMessageReceiver on ServiceBusTrigger

[MattJeanes]

I understand IMessageReceiver is no longer bindable with dotnet-isolated but what is the recommended way to achieve the functions it previously provided now? For example, manual completion of the message, receiving deferred messages, deferring messages etc. I cannot see any way to do this now.

[fabiocav]

Those scenarios are not currently supported. We'll continue to close those gaps as we enhance the binding model in the host and adopt that in the worker, but in the meantime, to use bindings for those scenarios, you still need to use the in-proc model.

[MattJeanes]

That is a shame, but thank you for the quick response. Is this planned for .NET 5 or is this more like something on the roadmap for .NET 6? Just would like to get an idea of a timeline as this is blocking our upgrade to .NET 5

[timgabrhel]

Another vote here. Our .NET 5 upgrade was dependent on the new isolated hosting model, but I'm now stuck for this same reason. We have two functions that are dependent on being able to complete/abandon + update message Properties on messages.

[fabiocav]

To provide an update and answer the previous question, this requires enhancements to the host and SDK so the Worker can leverage the new capabilities once introduced, and given the timelines, this will land in-proc in .NET 6 prior to being done in the worker.

A lot of activity will be happening in the coming weeks to make a .NET 6 preview available in a new major version of Functions.

[MatheusXavier]

I have a service where I can complete the messages, so I don't need the IMessegaReceiver, but I need the lockToken, Are there some way to get that? On documentation they talk about the MessageReceiver so I used this property with string type but I received a json with empty TokenProvider:

  "RegisteredPlugins": [
    
  ],
  "ReceiveMode": 0,
  "PrefetchCount": 100,
  "LastPeekedSequenceNumber": 0,
  "Path": "desk4me-monitoring-statuses-local-development",
  "OperationTimeout": "00:01:00",
  "ServiceBusConnection": {
    "Endpoint": "sb://desk4me-dev.servicebus.windows.net",
    "OperationTimeout": "00:01:00",
    "RetryPolicy": {
      "MinimalBackoff": "00:00:00",
      "MaximumBackoff": "00:00:30",
      "DeltaBackoff": "00:00:03",
      "MaxRetryCount": 5,
      "IsServerBusy": false,
      "ServerBusyExceptionMessage": null
    },
    "TransportType": 0,
    "TokenProvider": {
      
    },
    "IsClosedOrClosing": false
  },
  "IsClosedOrClosing": false,
  "OwnsConnection": true,
  "ClientId": "MessageReceiver1desk4me-monitoring-statuses-local-development",
  "RetryPolicy": {
    "MinimalBackoff": "00:00:00",
    "MaximumBackoff": "00:00:30",
    "DeltaBackoff": "00:00:03",
    "MaxRetryCount": 5,
    "IsServerBusy": false,
    "ServerBusyExceptionMessage": null
  }
}

[timgabrhel]

@MatheusXavier The lockToken can be found in the binding metadata off of the FunctionContext passed into the function. I tried to init my own SB connection and mark the message as complete with the lock token, but service bus rejects it as the message was received by another MessageReceiver. Curious if you have an alternative to make it work.

[MatheusXavier]

@timgabrhel I tried your solution, so I could get lockToken from BindingContext:
var lockToken = context.BindingContext.BindingData.GetValueOrDefault("LockToken").ToString();
I discovered that if you add a parameter on your function called string lockToken you could get it too, but when I tried to complete the message I got the same error:

{"The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance."}

I built the Microsoft.Azure.ServiceBus.QueueClient with all my Service Bus and Queue connection settings and then called the CompleteAsync but it didn't work. If I get this QueueClient and register a message handler, and inside it I use the CompleteAsync It works fine.

[timgabrhel]

@MatheusXavier That's great to hear! Any chance you can share a code snippet around the message handler & completing the message?

[MatheusXavier]

Sure! I use two differents queues on my application, the first one I use to handle events from my domain, on application startup I register a message handler that receives events, it is working perfectly, this is the code:

        public void StartConsumer()
        {
            _serviceBusConnection.GetQueue().RegisterMessageHandler(
                async (message, token) =>
                {
                    var messageData = Encoding.UTF8.GetString(message.Body);

                    // Complete the message so that it is not received again.
                    if (await ProcessEventAsync(message.Label, messageData))
                    {
                        await _serviceBusConnection.GetQueue().CompleteAsync(message.SystemProperties.LockToken);
                    }
                },
                new MessageHandlerOptions(ExceptionReceivedHandler)
                {
                    MaxConcurrentCalls = 10,
                    AutoComplete = false,
                });
        }

The second queue I use to process some data, so I use an Azure Function with Service Bus Trigger, this is the code on dotnet 3.1 version:

    public class Processor
    {
        private readonly IMediator _mediator;

        public Processor(IMediator mediator)
        {
            _mediator = mediator;
        }

        [FunctionName("processor")]
        public async Task Run(
            [ServiceBusTrigger("%QueueName%", Connection = "ServiceBusConnection")]
            Message message,
            ILogger log,
            MessageReceiver messageReceiver)
        {
            try
            {
                var jsonData = Encoding.UTF8.GetString(message.Body);
                var settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All };
                var deserialized = JsonConvert.DeserializeObject(jsonData, settings);

                await _mediator.Send(deserialized);
                await messageReceiver.CompleteAsync(message.SystemProperties.LockToken);
            }
            catch (Exception ex)
            {
                log.LogError(ex, $"[Processor] - An error with message: {ex.Message} occurred while " +
                    $"handling message with Id: '{message.MessageId}' and label: '{message.Label}' on {ex.StackTrace}");
            }
        }
    }

After the migration to dotnet 5 we don't have the MessageReceiver anymore, so I tried to use my _serviceBusConnection instance to complete the messages like the first one sample. This is the code:

    public class Processor
    {
        private readonly IMediator _mediator;
        private readonly IServiceBusPersisterConnection _serviceBus;

        public Processor(
            IMediator mediator,
            IServiceBusPersisterConnection serviceBus)
        {
            _mediator = mediator;
            _serviceBus = serviceBus;
        }

        [Function("processor")]
        public async Task Run(
            [ServiceBusTrigger("%QueueName%", Connection = "serviceBusConnection")] string message,
            string messageReceiver,
            string systemProperties,
            string messageId,
            string lockToken,
            FunctionContext context)
        {
            try
            {
                var settings = new JsonSerializerSettings { TypeNameHandling = TypeNameHandling.All };
                var deserialized = JsonConvert.DeserializeObject(message, settings);

                await _mediator.Send(deserialized);
                await _serviceBus.GetQueue().CompleteAsync(lockToken);
            }
            catch (Exception ex)
            {
                var log = context.GetLogger<Processor>();

                log.LogError(ex, $"[Processor] - An error with message: {ex.Message} occurred while " +
                    $"handling message with Id: '{messageId}' on {ex.StackTrace}");
            }
        }
    }

But I'm gettins this error on complete:

{"The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue, or was received by a different receiver instance."}

Any ideia to solve that?

[timgabrhel]

@MatheusXavier My apologies, I misunderstood and thought you meant you somehow were able to register a new message handler within the function to complete the message. Looks like we're on hold until this support is enabled by the functions team.

[Prinsn]

Is there any update on this?

This needs to be added to breaking changes, or some kind of guide needs to be written for how to get 3.x functions working with 5.0.

From all the research I have done today, it appears that Azure Functions hard stop an upgrade to .NET Core 5.0.

Am I mistaken in this understanding?

The fact that there is on official response on how to mitigate or migrate away from this is honestly damning.

[MattJeanes]

@Prinsn we had to dual target some of our library projects in the end and hold back upgrading most of our Microsoft.Extensions packages in order to upgrade to .NET 5.0 for our APIs while leaving the functions at .NET Core 3.1 without breaking either. It was a huge pain though, so I'm looking forward to Azure Functions being upgraded properly for .NET 6.

[Prinsn]

@MattJeanes 🙏 Is there any guide or anything you follow or something you could indicate as to how to manage it? Our project requires the app be 5.0, and we are currently troubleshooting our functions and I cannot find any mention of how to differentially modify the functions to support this.

[MattJeanes]

@Prinsn I didn't follow any guides myself I'm not aware of any. It was pretty much a case of changing target framework to net5.0 for our console apps / APIs but leaving the Azure Functions at netcoreapp3.1 and ensuring any library projects shared between them were netstandard2.1 or dual targetting net5.0 and netcoreapp3.1.

NuGet packages in the library projects especially from Microsoft had to be kept at 3.x such as Microsoft.Extensions.Logging as upgrading them breaks Azure Functions due to the in-process hosting model - which is incidentially one of the biggest reasons to move to out-of-process hosting using dotnet-isolated but it's just simply not feature complete for our needs yet, and a lot of work to change everything to use that as well.

In another fun twist of events though some Microsoft packages have to be upgraded to 5.x in the web projects such as Microsoft.AspNetCore.Mvc.NewtonsoftJson or that will break the APIs 😄

I wish you the best of luck, make sure to test them well after upgrading!

[Prinsn]

@MattJeanes just to clarify, from the sounds of the Web project needing things, which is technically unrelated to the Azure Function project, you had to keep some things at 3.x if they were merely referenced?

[MattJeanes]

@Prinsn yeah so for Microsoft.AspNetCore.Mvc.NewtonsoftJson for example that was referenced directly by the web project as 5.x which is fine, it's the NuGet packages used in your library projects like Microsoft.Extensions.Logging or Microsoft.Extensions.Configuration or Microsoft.EntityFrameworkCore etc that need to stay at 3.x if those libraries are referenced by your Azure Functions projects as it will break them.

We had an additional objective of ensuring package versions were consistent across the solution which is what also what drove our approach. We didn't want to say use [email protected] in one library project that wasn't referenced by an Azure Functions but [email protected] in another one that was, it would become a nightmare real fast.

I'd share the VS solution with you if I could but it's not open source and I'd definitely get into a lot of trouble! 😄

Below is a PowerShell script I wrote to upgrade all packages to their latest versions across a solution, except for ones that should stay at 3.x. It's not perfect but it might help you or others who take this approach:

$ErrorActionPreference = "Stop"
function Get-SemVer {
    param(
        [Parameter(Mandatory = $true)]
        [string]$Version
    )
    
    $semverRegex = "^(\d+)\.(\d+)\.(\d+)\.?(\d+)?-?(.+)?"
    if ($Version -match $semverRegex) {
        return @{ Version = $Matches[0]; Major = [int]$Matches[1]; Minor = [int]$Matches[2]; Patch = [int]$Matches[3]; Patch2 = [int]$Matches[4]; PreRelease = $Matches[5] } 
    }
    else {
        Write-Debug "Could not get semver from version '$Version'"
        return $null
    }
}

function Get-NugetLatestPackage {
    param(
        [Parameter(Mandatory = $true)]
        [string]$PackageName,

        [Parameter(Mandatory = $false)]
        [int]$MajorVersion,

        [Parameter(Mandatory = $false)]
        [switch]$IncludePreRelease
    )
    $versions = @()
    $res = Invoke-RestMethod "https://api.nuget.org/v3/registration5-gz-semver2/$($PackageName.ToLower())/index.json"
    $versions += $res.items.items.catalogEntry | Where-Object { $_.listed } | ForEach-Object { $_.version }
    $res.items.'@id' | Where-Object { -not $_.Contains("#") } | ForEach-Object {
        Write-Debug "Calling $_"
        $items = Invoke-RestMethod $_
        $versions += $items.items.catalogEntry | Where-Object { $_.listed } | ForEach-Object { $_.version }
    }
    $selectedVersion = $versions `
    | Where-Object { $_ -and (Get-SemVer $_) }
    | ForEach-Object { Get-SemVer $_ }
    | Sort-Object Major, Minor, Patch, Patch2, PreRelease -Descending `
    | Where-Object { (-not $MajorVersion -or ($MajorVersion -eq $_.Major)) -and (-not $_.PreRelease -or $IncludePreRelease) } `
    | Select-Object -First 1
    return $selectedVersion
}

$currentPackages = @{}
$files = Get-ChildItem -Exclude "(exclude-any-projects-here)" | Get-ChildItem -Include "*.csproj" -Recurse
$files | ForEach-Object {
    $xml = [xml](Get-Content -Raw $_.FullName)
    $xml.Project.ItemGroup.PackageReference | Where-Object { $_.Include -and $_.Version } | ForEach-Object {
        $currentPackages[$_.Include] = Get-SemVer $_.Version
    }
}

$ignorePackages = @{
}

$majorVersionLocks = @{
    "^Microsoft\.Extensions" = 3
    "^Microsoft\.AspNetCore" = 3
    "^Microsoft\.EntityFrameworkCore" = 3
    "^System\.ComponentModel\.Annotations" = 4
    "^Toolbelt" = 3
}

$newPackages = @{}
$currentPackages.Keys | ForEach-Object {
    $package = $_
    if ($ignorePackages[$package]) {
        Write-Warning "Ignoring package $package"
        return
    }
    $currentVersion = $currentPackages[$package]
    $majorVersionLock = $majorVersionLocks.Keys | Where-Object { $package -match $_ }
    if ($majorVersionLock) {
        $newVersion = Get-NugetLatestPackage $package -MajorVersion $majorVersionLocks[$majorVersionLock]
    }
    else {
        $newVersion = Get-NugetLatestPackage $package
    }
    if (-not $newVersion) {
        Write-Error "Unable to find latest version for $package"
    }
    if ($currentVersion.Version -ne $newVersion.Version) {
        Write-Host "Upgrading $package from $($currentVersion.Version) to $($newVersion.Version)"
    }
    $newPackages[$package] = $newVersion
}

$files | ForEach-Object {
    $content = (Get-Content -Raw $_.FullName)
    $newContent = $content
    $newPackages.Keys | ForEach-Object {
        $newVersion = $newPackages[$_]
        $match = "<PackageReference Include=`"$_`" Version=`"(.+?)`""
        if ($newContent -match $match -and $Matches[1] -ne $newVersion.Version) {
            Write-Debug "Upgrading $_ $($Matches[1]) -> $($newVersion.Version)"
            $newContent = $newContent -replace $match, "<PackageReference Include=`"$_`" Version=`"$($newVersion.Version)`""
        }
    }
    if ($content -ne $newContent) {
        Set-Content $_.FullName $newContent
        Write-Host "Modified $($_.FullName)"
    }
}

[Prinsn]

This is nightmarish...

I hope that I can leverage your experience in this issue to at least validate the effort you went through.

@MattJeanes One last thing, however. Is this strictly for external assemblies?

We're currently referencing our internal security resource as a nuget, which is required to be 5.0 which requires EFCore at 5.0, which are executed on by the functions.

I can come up with ways to work around this in the most janktastic fashions, but I'm not sure where the line is drawn between assemblies if this wont fundamentally break everything if I can't completely decouple the functions from solution internal assemblies that reference 5.0.

Kind of fishing for how much Excedrin I should take before trying to fix this.

[MattJeanes]

@Prinsn if your internal assemblies are referencing Microsoft packages it could still be an issue.

We tried to upgrade EF Core to 5.0 as well and because EF Core 5.0 transiently references a bunch of Microsoft.Extensions packages @ 5.0 (you can see that here under dependencies: https://www.nuget.org/packages/Microsoft.EntityFrameworkCore) it caused the Azure Functions to completely break, so we have had to remain on EF Core 3.0 across our solution for now due to this. I also tried trying to manually downgrade the packages it references to 3.0 like Microsoft.Extensions.Logging but that just caused even more problems.

[Prinsn]

This is a literal dumpster fire, I appreciate your sanity checks in this.

[MattJeanes]

Yes it's exactly why dotnet-isolated Azure Functions are a great idea as it avoids all these issues, but it's simply not ready for prime-time when it's missing critical features like the IMessageReceiver with no alternative. Hopefully .NET 6 will fix all of our woes 😅

[Prinsn]

Unfortunately, I cannot wait 5 months, and downgrading to 3.x is not an option, so we are currently looking into solutions to hack around this.

My cohort is currently trying to figure out if he can create an API service to get an authenticated token to allow us to renew locks via a REST API call, rather than the message receiver.

I'm currently now considering how possible it is to completely remove the functions from our solution and create something of an SDK for decoupling everything to remove any potential referencing issues.

[Prinsn]

My cohort managed to get something promising together by using calls to the azure REST API directly that are promising at least with respect to the lockToken and refreshing locks. We're going to be testing how it works with deadletter queues tomorrow (since that's like a two step), I hope to have a strategy soon for how to work around the issue. I don't know if the REST API is robust enough to handle most common cases, but those were the only two things we used the MessageReceiver for.

…

On Wed, May 19, 2021 at 11:58 AM Matt Jeanes ***@***.***> wrote: Yes it's exactly why dotnet-isolated Azure Functions are a great idea as it avoids all these issues, but it's simply not ready for prime-time when it's missing critical features like the IMessageReceiver with no alternative. Hopefully .NET 6 will fix all of our woes 😅 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#293 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYR7G5KTZUI6VE5PFFO5S3TOPN3JANCNFSM4ZCNDPEA> .

-- Jeff Miller

[Prinsn]

Okay, so this isn't gonna be the cleanest. I asked the engineer that worked this through for something of an MVP, and what he did was take what he did, gut all our proprietary stuff, and then put in untested things that approximate something similar.

//TL;DR: We used the REST API for the ServiceBus (e.g.: https://docs.microsoft.com/en-us/rest/api/servicebus/renew-lock-for-a-message)

Support for this is spotty since apparently it hasn't been updated since 2015, so pretty coooooool. You can't obviously interact with the deadletter queue (it says you can only do it based on the parent entity, and the API doesn't support it, so we didn't try), so it might require some work arounds to generate the same basic behavior for any manual deadlettering (i.e.: If you don't need it into DLQ proper and just need to log, you can basically just instantiate a new DLQ and pass in the stuff it would do and have it basically just operate as normal, as much as possible, so that it logs and excepts, then complete the message as normal so it is no longer on the queue)
//

His immediate comments were:

I hard coded the resourceUri, topic, and subscription strings. really, they should be pulled from your connection string (assuming it's of the Sas token variety)
basic MVP is, if you know the resourceUri of your bus, the topic, and subscription for your message, and the messge ID and lock, you can use those with a POST to renew the token indefinitely
all but the last two you should know by virtue of your function running, and the last two you can get from the message itself
the topic, and subscription and message id variables in that snippet are not defined
for the purposes of demonstration, they can be constants I suppose. the topic and subscription are strings that are needed in the input binding, so you have to have those. the resource Uri, key name, and key, should be in the connection string. the message id and lock are in the message

public class ServiceBusTopicTriggerCSharp1
{
    private readonly Timer lockRenewalTimer;

    public ServiceBusTopicTriggerCSharp1()
    {
        lockRenewalTimer = new Timer(1000 * 60); // renew lock every 60 seconds
    }
    [Function("ServiceBusTopicTriggerCSharp1")]
    public static async Task Run([ServiceBusTrigger("myTopicName", "mySubscription", Connection = "myConnectionString")] string mySbMsg, FunctionContext context, string messageId, string sequenceNumber, string lockToken)
    {
        var logger = context.GetLogger("ServiceBusTopicTriggerCSharp1");
        SetupAutoLockRenewal(messageId, lockToken);
        //function code here
    }
    protected void SetupAutoLockRenewal(string messageId, string lockToken)
    {
        if (lockRenewalTimer.Enabled)
        {
            return;
        }
        lockRenewalTimer.Elapsed += new ElapsedEventHandler(async (object sender, ElapsedEventArgs e) =>
        {
            using var client = new HttpClient();  //TODO: should be dependency injected as a singleton service to avoid port exhaustion issues

            var connectionString = "myConnectionString";  //or pull from settings/keyvault/environment/etc.  must be of the form  Endpoint=sb://{serviceBusResourceUri};SharedAccessKeyName={myKeyName};SharedAccessKey={myKey};TransportType=AmqpWebSockets;
            var resourceUri = connectionString.GetStringBetween("Endpoint=sb://", ";", StringComparison.InvariantCultureIgnoreCase);  //or retrieve/set manually
            var keyName = connectionString.GetStringBetween("SharedAccessKeyName=", ";", StringComparison.InvariantCultureIgnoreCase);  //or retrieve/set manually
            var key = connectionString.GetStringBetween("SharedAccessKey=", ";", StringComparison.InvariantCultureIgnoreCase);  //or retrieve/set manually
            var token = CreateToken(resourceUri, keyName, key);
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("SharedAccessSignature", token);

            var url = $"https://{resourceUri}/{topic}/subscriptions/{subscription}/messages/{messageId}/{lockToken}";
            var response = await client.PostAsync(url, null).ConfigureAwait(false);
            if (response.IsSuccessStatusCode)
            {
                //"Message lock successfully renewed"
            }
            else
            {
                //"Message lock renewal failed"
            }
        });
        lockRenewalTimer.Start();
    }
    private static string CreateToken(string resourceUri, string keyName, string key)
    {
        TimeSpan sinceEpoch = DateTime.UtcNow - new DateTime(1970, 1, 1);
        var week = 60 * 60 * 24 * 7;
        var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + week);
        string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
        using var hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
        var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
        var sasToken = string.Format(CultureInfo.InvariantCulture, "sr={0}&sig={1}&se={2}&skn={3}", HttpUtility.UrlEncode(resourceUri), HttpUtility.UrlEncode(signature), expiry, keyName);
        return sasToken;
    }
    private static string GetStringBetween(this string str, string startAfter, string endAtNext, StringComparison comparisonType)
    {
        int from = str.IndexOf(startAfter, comparisonType) + startAfter.Length;
        int to = str.IndexOf(endAtNext, from, comparisonType);

        return str[from..to];
    }
}

Our only two use cases for the MessageReceiver were to renew the lock and call DeadLetterQueueAsync, so hopefully it at least demonstrates how to interact with the REST API instead of via the Message Receiver for the cases that are supported by the REST API.

Also, it'd be rel kul if the REST API could be updated for the first time in 6 years to cover the gap.

[Prinsn]

Currently seems that this only works when developing locally, the REST API indicates correct lock renewal but doesn't appear to correctly hold the message, and the whole thing is moot as we're seeing database performance degradations almost to an order of magnitude.

It works in some functions but not others

[Prinsn]

@fabiocav Could you explain how this was triaged as an enhancement?

If all developers blocking on this cannot be instructed how to work around this issue to not block adoption of .NET 5.0, this would seem more than an enhancement.

[djfoxer]

@SeanFeldman I have:

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net5.0</TargetFramework>
  </PropertyGroup>

So it's In-Process as I assume now. Thank you for your patience.

[SeanFeldman]

@djfoxer look for the SDK package referenced.
Microsoft.NET.Sdk.Functions - In-Proc
Microsoft.Azure.Functions.Worker.Sdk - Isolated Worker

[djfoxer]

@SeanFeldman Ok, it's more complicated as It's WebJob: "Microsoft.Azure.WebJobs" 😁

[SeanFeldman]

@djfoxer, you're using InProc, mate.

Let's stop this discussion and return the thread to the original topic - IMessageReceiver on ServiceBusTrigger.

[aliqaryan]

these may explain my problem in a better way:

• [Azure Function ServiceBusTrigger] ServiceBusReceiveActions Azure function binding parameter missing PeekMessages method. azure-sdk-for-net#30887
• [Service Bus Extension] PeekMessage method on ServiceBusReceiveActions azure-sdk-for-net#29313

[sand-head]

any update on this? with the announcement that .NET Framework 4.8 can now use the v4 runtime in the out-of-process model, I've been looking at upgrading one of our v1 function apps, but the lack of support for taking manual action on a message is rather surprising

[emiledawalibi]

Would be very interested in an update on this given .NET 7 has now gone GA with v4 isolated. Being able to bind 'ServiceBusReceivedMessage' and 'ServiceBusMessageActions' on the ServiceBusTrigger (along with the App Insights issues) are the only things stopping me from migrating to isolated.

[hbrotan]

Currently on .NET 7.0, and I need a way to explicitly move a message to the deadletter queue, which IMessageReceiver supports for in-process, but there seems to be no support for this for isolated functions.

@fabiocav What are the plans to support this for isolated functions? This issue is > 1.5 years old and I would really appreciate some communication from Microsoft on this matter. Moving to isolated functions early has been a rough ride.

[aliqaryan]

Currently on .NET 7.0, and I need a way to explicitly move a message to the deadletter queue, which IMessageReceiver supports for in-process, but there seems to be no support for this for isolated functions.

@fabiocav What are the plans to support this for isolated functions? This issue is > 1.5 years old and I would really appreciate some communication from Microsoft on this matter. Moving to isolated functions early has been a rough ride.

in V4 all the required jobs can be done using these two input params:

            ServiceBusMessageActions messageActions,
            ServiceBusReceiveActions receiveActions,

[SeanFeldman]

@aliqaryan2 those are In-Process SDK. This repo/issue is about Isolated Worker SDK.

If it was that trivial, there's be no that many comments 🙂

[albernhagen]

I'm also encountering the same issues above where I am looking for more control around when I can dead letter and complete messages programmatically with the isolated model. Are there plans to support them any time soon?

[avisra]

This thread is unbelievable. I went on the journey to update my functions to .NET 7. Quickly abandoned that idea given how lacking the support is with Isolated Functions in .NET 7. Why remove in-proc support in .NET 7, if .NET 7 in-proc support doesn't have feature parity?

[MattJeanes]

There was a blog post back in September I've just found with some new information we've not seen in this thread yet: https://techcommunity.microsoft.com/t5/apps-on-azure-blog/net-on-azure-functions-roadmap-update/ba-p/3619066

In short, in-process Azure functions will be released only with LTS .NET releases (6, 8, 10, etc) and isolated will be released with every .NET version until isolated has achieved feature parity with in-process at which point they will stop releasing updates for in-process.

They previously gave a timeline implying .NET 6 would be the last in-process release but in the blog post they say this is no longer true, here is an updated graphic:

[hbrotan]

@fabiocav @kshyju Any update here? When can we expect to be able to explicitly move a message to the deadletter queue?

[philipborg]

This is a very critical issue for my team. Any indication that this is either worked on or when it's planned would be highly appreciated. Otherwise we may be forced to reconsider our usage of Azure functions and Service Bus entirely as Azure in-process is causing us to loose several man hours per week in just adding and maintaining workarounds to the plethora of issues we encounter because of it. Workarounds with severe downsides to our overall solution.

[dean-dot-git]

For those of us using service bus triggers in Azure Functions, this is a critical feature parity issue. Maintaining a new code base for all other functions while holding this one back doesn't make sense. Please consider this another plea to get this on the docket to be worked on. To be clear, need this for both standard and session-based message queues!

Thanks Microsoft for paying attention and listening to us!

[jeroenbongers]

I agree with all the others here, this is a blocking issue for us. We started upgrading our functions to isolated in .NET 7 but had to revert all our work after some time since we need this functionality.

I'm glad the blog post linked above gives us some hope that .NET 8 isolated either supports it, or still supports in-process, because it would have been a major issue for us going forward.

[jameswoodley]

This is crazy that this isn't supported, considering all the comms saying from .net7 onwards, the only model was isolated... I notice above that stance has changed, but I'd not seen any comms around this change. Currently isolated is pretty useless for the service bus trigger

[s-bauer]

It's just insane that this isn't fixed yet. I was so happy to finally be able to use isolated functions for the first time and not worry about dependency conflicts and stuff like this again. But with critical features like this missing, I need to go back to the old in-process model...

[syazdian]

If you see the first message in this thread is March 2021, and now is July 2023. Almost 2.5 years.
Do they have any updates that this would be fixed in .net 8?

[Archomeda]

I doubt it. At this point they'll still have to support in process for .NET 10 as well...

[timmkrause]

@Archomeda Really? @MattJeanes comment above confused me and so we opened a ticket with Microsoft. The primary aspects of the ticket are related to the roadmap (as I have reached a point where I don't know what we can trust/rely on anymore...) and the missing ServiceBusTrigger support in the isolated process model. We got the following response and I'd like to especially highlight the first sentence:

I have checked with team concerned .net 8 will be in only isolated mode.The team is working on closing gap between in-proc and out of proc models. For logging, the dotnet worker currently support ILogger logging similar to how you do it in a .net core app. You need to add the "https://www.nuget.org/packages/Microsoft.Azure.Functions.Worker.ApplicationInsights" package and enable logging in the Main of your app. See sample here in the PR description (#944)
If you think the isolated model is missing a feature you had in in-proc, I strongly suggest you open an issue in the worker repo. https://github.com/Azure/azure-functions-dotnet-worker/issues

Azure/azure-functions-host#9340 (comment)
#1641 (comment)
Service bus is one extension the team is currently working on closing the gap
The work is in-progress., https://github.com/Azure/azure-functions-dotnet-worker/blob/feature/sdk-type-binding/samples/WorkerBindingSamples/ServiceBus/ServiceBusReceivedMessageBindingSamples.cs
Can you ask that question in #1670 ?
The issue you have told is getting tracked on IMessageReceiver on ServiceBusTrigger · Issue #293 · Azure/azure-functions-dotnet-worker · GitHub
Please let me know if you have any further questions ?

If the ServiceBusTrigger support will not be added and it's isolated model only with .NET 8 Microsoft is basically telling us that we are (again!) stuck on .NET 6 and will have to wait AGAIN for .NET 9. This whole function story does not build trust on our end in the last 1-2 year,...

Before that our upgrade was blocked by the scoped logging issues which were not being addressed a long time...

[Archomeda]

They already confirmed that in process will be supported in .NET 8 as well for now if they can't reach feature parity with isolated.

[liliankasem]

This work is in progress as part of the SDK Type bindings effort. Issue (epic) #1081 tracks the work in Core (including host work) and the various extensions, with Service Bus included.

The initial updates for service bus have been released in preview and are described in the following issues:

• [SDK-Bindings - ServiceBus] Implement deferred binding support in WebJobs Extension #1536
• [SDK-Bindings - ServiceBus] Implement ServiceBus converter to bind to SBReceivedMessage  #1535

While the above does enhance the experience and provides support for some of the simpler SDK types, they don't provide support for message actions yet (any APIs that would require communication with the service such as MessageReceiver).

The following issue tracks the API work and should be coming in the next couple updates. This issue has a dependency on host enhancements being released first which are actively being worked on.

• Enhance Service Bus binding to enable calling service APIs #1538

As we have other issues tracking this work, I will be closing this one to reduce the number of duplicates we have open; feel free to refer to #226 for further updates.

[Pieeer1]

For those looking for a current workaround for IMessageReceiver and custom abandoning/completing messages,

In your dependency injections you can create any method of storing the receiver for specific topic and subscriptions:

        Dictionary<(string connection, string topic, string subscription), ServiceBusReceiver> serviceBusReceivers = 
            new Dictionary<(string connection, string topic, string subscription), ServiceBusReceiver>()
        {
            { 
                ("SpecificConnection", "SpecificTopic", "SpecificSub"),
                new ServiceBusClient(configuration["SpecificConnection"])
                    .CreateReceiver(configuration["SpecificTopic"], configuration["SpecificSub"])
            }
        };

        services.AddSingleton(serviceBusReceivers);

I use the tuple dictionary to quickly spin this up there are 1000% more readable ways to do this

    Dictionary<(string connection, string topic, string subscription), ServiceBusReceiver> _serviceBusReceivers;
    private readonly ILogger<CommunicationListener> _logger;

    public CommunicationListener(Dictionary<(string connection, string topic, string subscription), ServiceBusReceiver> serviceBusReceivers, ILoggerFactory logger)
    {
        _serviceBusReceivers = serviceBusReceivers;
        _logger = logger.CreateLogger<CommunicationListener>();
    }

    [Function(nameof(CommunicationListener))]
    public async Task Run([ServiceBusTrigger("%SpecificTopic%", "%SpecificSub%", Connection = "SpecificConnection")]
    ServiceBusReceivedMessage message)
    {
        await _serviceBusReceivers[("SpecificConnection", "SpecificTopic", "SpecificSub")].CompleteMessageAsync(message);
        //or
        await _serviceBusReceivers[("SpecificConnection", "SpecificTopic", "SpecificSub")].AbandonMessageAsync(message);
    }

[SeanFeldman]

@Pieeer1, are you sure about that? You're registering a receiver that's not the same as the function used to retrieve the message. You cannot settle the message because it's not the same receiver.

[Pieeer1]

@Pieeer1, are you sure about that? You're registering a receiver that's not the same as the function used to retrieve the message. You cannot settle the message because it's not the same receiver.

Ah I stand corrected :/ really unfortunate issue then.