How to get AppInsights working in .NET 8 Functions v4

[LockTar]

Hi,

Sorry for the new issue but I think this will bring a lot of open issues together to a final solution in a sample repository.
Hopefully we can make something default/officially/easier for new people because it took me a long time to figure everything out.

• This setup is ugly but solves AppInsights logging in Azure Functions v4 in combination with .NET 7. It uses the new (still in preview) Microsoft.Azure.Functions.Worker.ApplicationInsights package explained in PR 944
• It fixes the hardcoded Warning filter explained in 1123 comment. See auto added Warning filter details here.
• It highlights that the default Warning filter is automatically added explained here.
• It adds an appsettings.json configuration file for default configuration as suggested in Update samples and docs to add Host.CreateDefaultBuilder() #1025
• It loads the appsettings.json configuration file.
• It configures the logging from out of the appsettings.json configuration file
• It is a more realistic sample application other than "Hello world" because of multiple classes, loggers and injected loggers.

I want to thank @brettsam, @jviau and @kshyju for posting helpful comments that let me finally see some logging in Azure Application Insights lower than Warning (so Trace, Debug, Information).

See the sample repository for more details to get it running. See the details of the Func.Isolated.Net7.With.AI project.

Screenshots from Azure:

[SeanFeldman]

@LockTar, is that repo public?

[LockTar]

@LockTar, is that repo public?

Oeps! Forgot to change that. It's public now

[LockTar]

@SeanFeldman
Basically it results in the following:

1. Add the 2 new lines of the preview package
2. Remove the warning and above logger
3. Use appsettings.json as configuration file
4. Configure logging from appsettings.json
5. Optional setup DI for testing injected classes with ILogger<T>

Program.cs

using Microsoft.Extensions.Hosting;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Func.Isolated.Net7.With.AI;

var host = new HostBuilder()
    .ConfigureFunctionsWorkerDefaults(builder =>
    {
        // Is added by package "Microsoft.Azure.Functions.Worker.ApplicationInsights".
        // Documented here because it is still preview: https://github.com/Azure/azure-functions-dotnet-worker/pull/944#issue-1282987627
        builder
            .AddApplicationInsights()
            .AddApplicationInsightsLogger();        
    })
    .ConfigureServices((ctx, serviceProvider) =>
    {
        // You will need extra configuration because above will only log per default Warning (default AI configuration) and above because of following line:
        // https://github.com/microsoft/ApplicationInsights-dotnet/blob/main/NETCORE/src/Shared/Extensions/ApplicationInsightsExtensions.cs#L427
        // This is documented here:
        // https://github.com/microsoft/ApplicationInsights-dotnet/issues/2610#issuecomment-1316672650
        // So remove the default logger rule (warning and above). This will result that the default will be Information.
        serviceProvider.Configure<LoggerFilterOptions>(options =>
        {
            var toRemove = options.Rules.FirstOrDefault(rule => rule.ProviderName
                == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");

            if (toRemove is not null)
            {
                options.Rules.Remove(toRemove);
            }
        });


        // Setup DI
        serviceProvider.AddTransient<IUserDataService, UserDataService>();
    })
    .ConfigureAppConfiguration((hostContext, config) =>
    {
        // Add appsettings.json configuration so we can set logging in configuration.
        // Add in example a file called appsettings.json to the root and set the properties to:
        // Build Action: Content
        // Copy to Output Directory: Copy if newer
        //
        // Content:
        // {
        //   "Logging": {
        //     "LogLevel": {
        //       "Default": "Error" // Change this to ie Trace for more logging
        //     }
        //   }
        // }
        config.AddJsonFile("appsettings.json", optional: true);
    })
    .ConfigureLogging((hostingContext, logging) =>
    {
        // Make sure the configuration of the appsettings.json file is picked up.
        logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
    })
    .Build();

host.Run();

Add appsettings.json file with settings Build Action: Content and Copy to Output Directory: Copy if newer:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Func.Isolated.Net7.With.AI.MyOtherFunctions": "Error",
      "Func.Isolated.Net7.With.AI.MyUserFunctions": "Debug",
      "Func.Isolated.Net7.With.AI.UserDataService": "Trace"
    }
  }
}

[jviau]

@LockTar thank you for summarizing all of this for others to reference. Glad you were able to get it working. Was the logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging")); call what we were ultimately missing in the previous discussions? If so, I wonder if the Logging:ApplicationInsights:LogLevel:Default = Information would actually work with that? This would replace the need for the Configure<LoggerFilterOptions> hack.

Anyways - is there any action items you want from our side as an outcome of this issue? Keep in mind we do already have intentions to address telemetry in Functions (ideally go with OpenTelemetry).

[LockTar]

Was the logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging")); call what we were ultimately missing in the previous discussions? If so, I wonder if the Logging:ApplicationInsights:LogLevel:Default = Information would actually work with that? This would replace the need for the Configure<LoggerFilterOptions> hack.

Just tested it for you.
If I don't set the delete logger filter rule from you and change appsettings.json to:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Func.Isolated.Net7.With.AI.MyOtherFunctions": "Error",
      "Func.Isolated.Net7.With.AI.MyUserFunctions": "Debug",
      "Func.Isolated.Net7.With.AI.UserDataService": "Trace"
    },
    "ApplicationInsights": {
      "LogLevel": {
        "Default": "Debug"
      }
    }
  }
}

Then I would see Debug messages as well. So, this would change the default warning rule.

BUT
If I change it to Logging:ApplicationInsights:LogLevel:Default = Error, then I of course only see Error messages but I'm losing in example the Logging:LogLevel:Func.Isolated.Net7.With.AI.UserDataService = Trace messages.

That's not what you want... Maybe you have fix for that as well but that is exactly the thing that everyone is saying. It is to unclear how everything is working for Azure (Isolated) Functions.

[LockTar]

Anyways - is there any action items you want from our side as an outcome of this issue? Keep in mind we do already have intentions to address telemetry in Functions (ideally go with OpenTelemetry).

Yes.

1. My console logging (localhost debugging) is missing completely. How can this be fixed... Is this related with Logging calls does not appear in console when running locally in 1.9.0 using ApplicationInsights preview packages #1090?
2. I/We want an official sample application or new VS2022 default project template with clear documentation on how to set logging. Now we have 3 places to set logging and it's unclear how everything is working. We have host.json, appsettings:Logging:LogLevel and appsettings:Logging:ApplicationInsights:LogLevel.
   Everything is based on assumptions.

I don't know if OpenTelemetry is going to fix everything but I guess this will not be finished tomorrow. We create new Azure Functions on almost daily level. We need to have clear logging (documentation) from the start.

[jviau]

For the log filters: Logging:ApplicationInsights:LogLevel:Default = Error and Logging:LogLevel:Func.Isolated.Net7.With.AI.UserDataService = Trace only showing Error in App Insights is behaving as expected. The first one is a provider specific filter, which takes precedence over the second filter you have. This reminds me why I have that filter removal 'hack' in my code and not the config approach. It is the same situation as you: I wanted to control log filters through the non-provider configuration (Logging:LogLevel:*). App insights package always adding that Warning filter gets in the way, so I removed it. The hack isn't the best, but also not the worst. We will not be porting that into our app insights package as we do not want to conflict with what the AppInsights package has decided to do. We will leave the decision to include that filter customization to the end user.

For the console logging missing completely in local debugging, something I was missing this entire time was that stdout for the worker is piped to the host, which is not always forwarding them to the terminal you are running in. @fabiocav / @brettsam what is the recommended way to get console logs from worker in local debugging?

And we do intend for the OpenTelemetry work to address all of these concerns. You will have much more control over the logs. However, something that won't change is the multiple places to configure logging. Ultimately, you have two processes running - the host and the worker and they have their own respective logging configuration. We will always have to expose the ability to configure them separately (host.json for the host config, then you are ultimately in control of the setting file(s) for the worker). As for the multiple "LogLevel" sections to configure, that is also out of our control. This is how Microsoft.Extensions.Logging works - we do not own that API or its configuration.

To summarize, sounds like we just need to help you with the worker --> host stdout pipe causing issues with console logging.

[LockTar]

App insights package always adding that Warning filter gets in the way, so I removed it. The hack isn't the best, but also not the worst. We will not be porting that into our app insights package as we do not want to conflict with what the AppInsights package has decided to do. We will leave the decision to include that filter customization to the end user.

I understand that you don't want to make conflicts with the AppInsights packages. Otherwise, you will get mixed results in MS Azure services. But removing the Warning filter could be added to the project template in VS20xx. Maybe not enabled but in comment. Maybe with a link to the docs (that I can't find anywhere for Azure Functions...). (See below bullet 2)

And we do intend for the OpenTelemetry work to address all of these concerns. You will have much more control over the logs. However, something that won't change is the multiple places to configure logging. Ultimately, you have two processes running - the host and the worker and they have their own respective logging configuration. We will always have to expose the ability to configure them separately (host.json for the host config, then you are ultimately in control of the setting file(s) for the worker). As for the multiple "LogLevel" sections to configure, that is also out of our control. This is how Microsoft.Extensions.Logging works - we do not own that API or its configuration.

Thank you for the explanation. I understand that there are 2 types of places to configure.

To summarize, sounds like we just need to help you with the worker --> host stdout pipe causing issues with console logging.

1. Well, yes. Console logging would be nice for local debugging.
2. How can we make things clearer for people that create a new project. The appsettings.json and Program.cs as stated above aren't default. There is no clear documentation from an Azure Function Isolated perspective how to set this all up... It took me hours and I'm not the only one. It would be nice to have documentation about this and a changed project template.

Don't get me wrong (not attacking you or colleagues at all) but my employers want me to write (small) business solutions. Azure Functions are ideal for this. That's why I'm using Functions from the beginning. Logging should be more an addition for troubleshooting. It now takes me longer to figure out how logging works (it changes a lot) than I'm writing a complete API...

[drew-fc]

hi @LockTar ,

Thank you so much for putting this together.
I have integrated your changes into my project, but the result I am getting is that nothing is printed at all to the terminal during debugging sessions.
Previously, I was only getting Information and above to print.
I have downloaded your entire repo and tried to get your project to work, but it fails on host.Run() when it starts. The error I get is

`Exception has occurred: CLR/System.InvalidOperationException
An unhandled exception of type 'System.InvalidOperationException' occurred in System.Private.CoreLib.dll: 'The gRPC channel URI 'http://:7283' could not be parsed.'

Any ideas how to move forward?
I am using Visual Studio Code if that matters, not Visual Studio, although I have VS if you think that's absolutely needed to move forward.
thanks again

[LockTar]

@drew-fc

Thank you so much for putting this together.

Your welcome.

I have integrated your changes into my project, but the result I am getting is that nothing is printed at all to the terminal during debugging sessions. Previously, I was only getting Information and above to print.

I have exactly the same problem. See above comment 1 and comment 2 about this. I think this is related with #1090 but I want that answer from Microsoft because they understand this better than I do.

I have downloaded your entire repo and tried to get your project to work, but it fails on host.Run() when it starts.
Any ideas how to move forward? I am using Visual Studio Code if that matters, not Visual Studio, although I have VS if you think that's absolutely needed to move forward.

I never use Visual Studio Code with Azure Functions so I don't know if this is the issue. This is a normal new Azure Function template created with the latest version of Visual Studio 2022.

P.S. I think it's not a good idea to mix issues here. Could you please remove you stack trace from above because it's not really well formatted and has nothing to do with this issue. Thank you very much in advance.

[drew-fc]

@LockTar , thank you so much, I was able to get the console logs working with that fix of AddConsole().

[SeanFeldman]

Is there documentation by any chance to explain how the isolated worker functions work with logging, with some visualization? The sample @LockTar has put together is great, but I'd want to see this made into the official documentation. There are samples, and there are code snippets. There should be solid documentation to distill this down.

cc @fabiocav

[SeanFeldman]

I'm getting flooded with messages with the category Azure.Core at the Information log level despite configuring the default logging level to Error on both, host.json and appsettings.json. @LockTar, did you have to filter out that one?

[LockTar]

I'm getting flooded with messages with the category Azure.Core at the Information log level despite configuring the default logging level to Error on both, host.json and appsettings.json. @LockTar, did you have to filter out that one?

@SeanFeldman Is there a simple overview to see all the messages of a category?
I see a lot of messages with category Host etc. (opened a few of them).

I think the default (is it with a capital or lower d?) level is doing something. I get mixed results. Can't really figure out what it is. We need Microsoft for this.

[LockTar]

@drew-fc please create new issues for this and remove your comments. This has nothing to do with Azure Functions and logging. This is C# DI. I think StackOverflow is a better place for this question.

[LockTar]

My personal opinion is to inject loggers in the constructor. Just like this. How to inject multiple classes is a DI related question. That's why I said that. But I really think you should create a seperate issue for this and delete your comments (and then I can delete my comments) because this is really something else. Tag me in the other issue so I can maybe assist you. Thanks

[rellis-of-rhindleton]

This is what I'm getting from reading through all these comments and doing some experimentation. Please correct me if I'm wrong. Happy to edit/change/delete this comment to avoid confusion.

(edited: host.json and overrides)

• Logging can be configured as needed in the function app, e.g. with ConfigureLogging or by the use of an appsettings file as in the above examples.
• Log entries go out via the host. If the host has filters applied that won't pass log entries through, they get dropped, e.g. if host.json has minimum level of Information then you won't see any Debug entries come through no matter how the app itself is configured. (This might not apply with third party logging sinks e.g. via Serilog.)
• Likewise, if the host is not listening for log entries, they will be dropped. (This could be what is happening prior to host.Run() but I'm just speculating.)
• Logging configuration for the host (host.json) does not appear to follow the same rules you would expect in e.g. ASP.NET. In particular, category names for logging from functions are not class names; instead they are "Function.FunctionName" for entries generated by the host/framework, and "Function.FunctionName.User" for entries generated directly from code (logger.LogInformation). This is not obvious as category names are not shown in local development.
• Overrides for host.json work for some logging categories but not all. AzureFunctionsJobHost__logging__logLevel__Function works, but ...__Function__FunctionName__User does not.
• Adding the ApplicationInsights logging packages causes problems with seeing log entries in local development.
• In my own experiment (.NET 6, isolated process) the AddApplicationInsights* helpers do not appear to be necessary during local development. As long as an APPINSIGHTS_INSTRUMENTATIONKEY setting exists, log entries are making it to Application Insights (?!). They are also visible in the local host terminal.
• There is work planned to improve all this. There will be a focus on enabling/encouraging OpenTelemetry.

I'm just trying to understand what I'm dealing with. Having to jump back and forth between computers to write this so forgive me if I get a detail wrong.

[LockTar]

I think you are correct about everything. A lot is speculating so I'm not sure about everything. Hopefully Microsoft will bring some official documentation about this and a new project template. But there is not much response anymore about this...

Adding the ApplicationInsights logging packages causes problems with seeing log entries in local development.

This should solve it (not tested it myself).

[stevelknievel76]

@LockTar This thread has been very useful and enabled me to get my log levels working. Thanks very much. I have 1 outstanding issue though which I don't understand. Apologies if I did something stupid, but would be grateful if you can shed some light on it. Here is an example Invocation Log:

Notice that the "Type" column is not populated for all messages other than the first (that message was generated by the TimerTrigger framework, and not by my code).

When I look in traces I see that the customDimensions has a different format for the first message from all others (and does not include LogLevel):

... versus ...

Do you see this too?

[SeanFeldman]

@SeanFeldman Is there a simple overview to see all the messages of a category?
I see a lot of messages with category Host etc. (opened a few of them).

At this point, I'm not sure what's going on. Not only are the Azure SDKs at fault for logging multiple messages with the Information level, but also the number of categories (Azure.Core, Azure.Core.1, Azure.Messaging.ServiceBus, etc.). This is where a good, solid official sample is necessary, along with a proper documentation. Besides the noise in Application Insights, this will directly impact Azure bill clients will pay, and I'm concerned there's no good way to filter out the unnecessary logs.

@brettsam, @fabiocav, the issue with AppInsights and logging is becoming more of a pain with financial impact rather than just an inconvenience.

[drew-fc]

hey @LockTar , sorry to tag you, but I am following your example and I am stuck.
I am using the latest app insights which you linked above, and trying to get logs to work in my .NET 8.0 v4 function app.
My program.cs looks pretty much identical to this:https://github.com/devops-circle/Azure-Functions-Logging-Tests/blob/master/Func.Isolated.Net7.With.AI/Program.cs

However, everything is being written to AppInsights as "trace" with "Information" as the severity level. Here's an example:

_logger.LogTrace("TestLog Trace"); _logger.LogInformation("TestLog Information"); _logger.LogError( "TestLog Error"); _logger.LogError(new Exception("something wrong"),"TestLog ErrorWithException");

Shows up like this:

Any ideas?

[LockTar]

I haven't created any .NET 8 function yet. We are still on .NET 6 because of certain missing functionality. We should be able to upgrade to .NET 8 now but we have other priorities.

Ontopic:
Seems like it's changed again if you have followed everything correctly. When you download the sample and update to .NET 8 doesn't it work?
Is your appsettings.json picked up correctly?

I was hoping that Microsoft/Functions team already picked this GitHub issue up for better explanation/documentation and samples.

[drew-fc]

hey @LockTar , thanks for the reply.
I only ugpraded because Azure warned me that in a few months, .NET 7 function apps would be deprecated.
I supposed I can try to figure out to roll it back.
My Appsettings.json is a whole nother story. When using appsettings.json, it picks up every log including "trace" and "debug" but it writes them as "Information" level.
When I use host.json instead, it no longer picks up any "trace" or "debug" logs, but correctly identifies the severity level.
So I am stuck between deciding which type of Broken I want.
This is crazy as this is a very basic feature and I'm surprised no one else is complaining.

[drew-fc]

@LockTar looks like I found a hacky work-around to get this working.
Azure/azure-functions-host#8901
I had to manually set the minimum log level in ConfigureLogging
If this is "by design", I guess because host.json is log levels for the host and not for the function app, but how can we simply set the log level properly via configure without hard coding it like this?
I wrote some code to parse the host.json file to get the configured default value and set it there, but it seems very hack.
The AppSettings config version has the issue I shared in the screenshot, everything gets logged as Information, so I can't use appsettings.

[LockTar]

This issue is still valid for .NET 8. So, I changed the title. Hopefully this will be picked up? This important issue is already open since 2022...

[Fazer01]

@LockTar
Piece of art you put together here!

Following this, because we have the same issue here... The documentation is very unclear about how to achieve simple things as configure logging. More specifically what to define in the host.json file and what to define in the appsetttings.json.
Also the difference in var host = new HostBuilder()... versus var host = Host.CreateDefaultBuilder() (mentioned here: #1090 (comment)) which makes sure some defaults are loaded (appsettings.json file is added to the configurationpipeline etc.) is mentioned nowhere in the docs mentioned above.

@jviau maybe he can elaborate after a few years? :)

Thanks in advance!

[ydhondt]

I have been suffering with this for a couple days.

ConfigureFunctionsWebApplication() is now preferred over ConfigureFunctionsWorkerDefaults() and if you follow the rest of the examples as is, the filtering for ApplicationInsights breaks down again. This seems to have to do with the aspnetcore integration. ApplicationInsights configuration is handled differently there.

In the end, I found that there are now two paths, each requiring their own dependencies and ways of working.

	Without aspnet core integration	With aspnet core integration
Packages	Microsoft.ApplicationInsights.WorkerService Microsoft.Azure.Functions.Worker.Extensions.Http	Microsoft.ApplicationInsights.AspNetCore Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore
In Program.cs	ConfigureFunctionsWorkerDefaults()	ConfigureFunctionsWebApplication()
In Program.cs / ConfigureServices	services.AddApplicationInsightsTelemetryWorkerService()	services.AddApplicationInsightsTelemetry()

@brettsam Can you confirm this? And if so, get the documentation updated?

Something else I noticed is that when doing a build and deployment on docker, the worker tends to end up in a different folder than the host ( /home/site/wwwroot vs /azure-functions-host) in the basic templates. However, by default the application runs with the host folder as its base path. This has an impact on the excellent samples provided by @LockTar . The loading of the appsettings.json loads the 'default' appsettings.json in the host directory (/azure-functions-host) rather than the one in the worker directory. You can work around that issue by explicitly setting the base path based on the location of the worker assembly:

config.SetBasePath(Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location))
      .AddJsonFile("appsettings.json", optional: false)

[JuanGRomeo]

Hi @LockTar

I have found this thread after reaching mostly the same solution you mentioned in the sample repo and I would like to ask if it is correct or not to use an appsettings.json file, because in the end I am not using it.

I came to this similar issue after starting to migrate our Azure Functions .NET6 In-Process to .NET 8 isolated worker model, mostly like @drew-fc on his comment #1182 (comment) .

With .NET 6 our Functions are able to write logs on Application Insight without any issue, and we were also able to modify the LogLevel, by changing the Environment variables on the Function App in case some bug appears on any environment.

When I started to migrate them to .NET 8 on Isolated model I realized all logs were constantly written in Error LogLevel or higher. Then, I realized on this documentation: https://learn.microsoft.com/en-us/azure/azure-functions/dotnet-isolated-process-guide?tabs=windows#application-insights
that I had to remove a default LoggerFilterRule.

Once I applied that configuration, all logs started to be written in Information Level or higher.

The last point I needed was to setup the LogLevel based on environment variables that I could also modify on Azure If I needed without any deployment.

I achieved firstly with an appsetting.json file and the following configuration:

    .ConfigureAppConfiguration((context, config) =>
    {
        config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true);
        config.AddEnvironmentVariables();
    })
    .ConfigureLogging((context, logging) =>
    {
        logging.AddConfiguration(context.Configuration.GetSection("Logging"));
    })

But it was a little bit strange for me since we don't have any appsetting.json file on none of our .NET 6 Azure Functions. Also motivated by this two threads:

• https://stackoverflow.com/questions/77834609/what-is-the-point-of-having-an-appsettings-json-file-in-my-azure-function-app
• https://stackoverflow.com/questions/69732989/dotnet-core-azure-function-isolated-process-how-to-load-appsettings-json-per-e

And also by this documentation: https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-use-azure-function-app-settings?tabs=azure-portal%2Cto-premium

which is not suggesting to use an appsetting.json file, I tryed to remove that part and it's working.

So Finally, I have an example Function App which is able to write Logs on Application Insights in the LogLevel configured in an Environment Variable which I can modify without deploying the Function App. The code is mostly similar but removing the appsettings.json part:

Program.cs:

using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureAppConfiguration((context, config) =>
    {
        config.AddEnvironmentVariables();
    })
    .ConfigureLogging((context, logging) =>
    {
        logging.AddConfiguration(context.Configuration.GetSection("Logging"));
    })
    .ConfigureServices(services =>
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
        services.Configure<LoggerFilterOptions>(options =>
        {
            // The Application Insights SDK adds a default logging filter that instructs ILogger to capture only Warning and more severe logs.
            // Application Insights requires an explicit override.
            // Log levels can also be configured using appsettings.json.
            // For more information, see https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service#ilogger-logs
            LoggerFilterRule defaultRule = options.Rules.FirstOrDefault(rule => rule.ProviderName
                == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");

            if (defaultRule is not null)
            {
                options.Rules.Remove(defaultRule);
            }
        });
    })
    .Build();

host.Run();

Function1.cs:

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;

namespace AppInsight.Logs.Testing
{
    public class Function1
    {
        private readonly ILogger<Function1> _logger;

        public Function1(ILogger<Function1> logger)
        {
            _logger = logger;
        }

        [Function("Function1")]
        public IActionResult Run([HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequest req)
        {
            _logger.LogDebug("Testing Log Debug");

            _logger.LogTrace("Testing Log Trace");

            _logger.LogInformation("Testing Log Information");

            _logger.LogWarning("Testing Log Warning");

            _logger.LogError("Testing Log Error");

            _logger.LogCritical("Testing Log Critical");

            return new OkObjectResult("Welcome to Azure Functions!");
        }
    }
}

host.json

{
  "version": "2.0"
}

[LockTar]

I would like to ask if it is correct or not to use an appsettings.json file, because in the end I am not using it.

Well, that is more a choice for you, I guess. The whole idea of this issue is to get logging working and I personally think that you want to set the loglevel for specific parts of the application. Not everything to Trace and everything to Error. It's easy to use appsettings.(ENV HERE).json so it's the same as on other frameworks. You want to control logging in development/locally other than in production. Also, you want to have the settings more in source control (my opinion).

which is not suggesting to use an appsetting.json file, I tryed to remove that part and it's working.

I don't see the part that they don't suggest to use an appsettings.json. Or I'm missing your point.
My ideal situation would be is to use different levels.
I want to clone my project on a new machine (or we have a new team member), hit F5 and all my settings and log levels are good to go (of course not the secrets but that is a different story. Use for example a KeyVault). local.settings.json is not really that great. Also, hierarchy is also not possible.

So Finally, I have an example Function App which is able to write Logs on Application Insights in the LogLevel configured in an Environment Variable which I can modify without deploying the Function App. The code is mostly similar but removing the appsettings.json part.

Do you have a sample application for me? Or fork my repo and create a new project next to it? That would be great.

[Sefriol]

I am just getting so frustrated by hard configuring these logs are. So far I just want

1. Performance analytics from each executed function (for example the execution time of [Function(MyFunction)])
2. ILogger logs into traces

I have done like 50 iterations of different configurations, but either performance logs completely disappear or traces are full of every possible log item that you could possibly get from Azure Function.

So far my function looks like this, but I am completely unsure what I should configure at host.json/appsettings.json vs in the function code.

public class Program
{
    public static IHostBuilder GetHostBuilder()
    {
        return new HostBuilder()
            .ConfigureFunctionsWebApplication(builder => { builder.UseMiddleware<ExceptionHandlerMiddleware>(); })
            .ConfigureLogging((context, logging) =>
            {
                //logging.AddFilter("System.Net.Http.HttpClient", LogLevel.Warning);
                //logging.AddFilter("Microsoft", LogLevel.Warning);
            })
            .ConfigureServices((hostContext, services) =>
            {
                services.AddApplicationInsightsTelemetryWorkerService();
                services.ConfigureFunctionsApplicationInsights();
                services.Configure<LoggerFilterOptions>(options =>
                {
                    // The Application Insights SDK adds a default logging filter that instructs ILogger to capture only Warning and more severe logs.
                    // Application Insights requires an explicit override.
                    // Log levels can also be configured using appsettings.json.
                    // For more information, see https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service#ilogger-logs
                    LoggerFilterRule? defaultRule = options.Rules.FirstOrDefault(rule =>
                        rule.ProviderName == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");

                    if (defaultRule is not null)
                    {
                        options.Rules.Remove(defaultRule);
                    }
                });
            });
        }
    }

[Sefriol]

I was able to get what I wanted after doing some trial and error in local instance connected to Application Insights.

Added following

            .ConfigureLogging((context, logging) =>
            {
                logging.AddFilter<ApplicationInsightsLoggerProvider>("Microsoft", LogLevel.Warning);
                logging.AddFilter<ApplicationInsightsLoggerProvider>("Host", LogLevel.Warning);
                logging.AddFilter<ApplicationInsightsLoggerProvider>("Host.Results", LogLevel.Information);
                logging.AddFilter<ApplicationInsightsLoggerProvider>("System", LogLevel.Warning);
            })

and

// host.json
{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "default": "Warning"
    },
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      },
      "enableLiveMetricsFilters": true
    }
  }
}

EDIT: Nvm. Local implementation does not work in cloud.

EDIT2: alright took forever, but final working config for myself

{
  "version": "2.0",
  "logging": {
    "logLevel": {
      "Function.*": "Warning",
      "Host.Triggers": "Warning",
      "Microsoft": "Warning"
    },
    "applicationInsights": {
      "samplingSettings": {
        "isEnabled": true,
        "excludedTypes": "Request"
      },
      "enableLiveMetricsFilters": true
    }
  }
}

.ConfigureLogging((context, logging) =>
{
    logging.AddFilter<ApplicationInsightsLoggerProvider>("Microsoft", LogLevel.Warning);            
    logging.AddFilter<ApplicationInsightsLoggerProvider>("System", LogLevel.Warning);
})

Kinda explanation what everything does. In some cases you can add more explicit filters if you want to keep some specific information.

Function:

• Microsoft filters out all the AspNet Mvc messages which do not really provide anything meaningful at Information level
• System filters out all of our internal HttpClient timer logs which can be seen at performance telemetry

Host:

• Function.* filters out all the "Executed", "Executing" etc logs
• Host.Trigger also filter outs function trigger logs which can be seen in telemetry
• Microsoft filters out all the host level config etc messages

[progmars]

After reading all of this sad logging story, I'm now wondering where should applicationInsights settings (such as "enableDependencyTracking": false) be stored for isolated model?

It seems they don't work if I leave the settings in host.json only.
I copied them to appsettings.json too and I have logging.AddConfiguration(context.Configuration.GetSection("Logging")) but enableDependencyTracking seems to be ignored, I still see every call to Blob Storage being logged as dependency in Insights.

The fact that Function initialization differs so much from Web Apps (WebApplication.CreateBuilder vs new HostBuilder() with chained function calls) adds even more to the confusion. I wish the developers of Web Apps and Functions came together and created a unified approach that works the same way and loads logging settings from a single place, to avoid copy-pasting them everywhere with the hope that it will stick.

[wondertalik]

I tried all the solutions from here and nothing works when function is running in docker container and as result when application insights used in azure i can see only warning and highers logs. Can anybody has as example that works?

[springcomp]

In #2531, I illustrate how to reliably have full control on your logging experience. This also aligns with both the latest practices and versions of the NuGet packages.

For those who prefer using the IHostBuilder directly, here is the same code

using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

var host = new HostBuilder()
    .ConfigureFunctionsWebApplication()
    .ConfigureAppConfiguration((hostContext, config) =>
    {
        // Add appsettings.json configuration so we can set logging in configuration.
        // For instance, add a new file called "appsettings.json" to the root of your project
        // and set the following properties to:
        // Build Action: None
        // Copy to Output Directory: Copy if newer
        //
        // Content:
        // {
        //   "Logging": {
        //     "LogLevel": {
        //       "Default": "Warning" // Change this to i.e. "Trace" for more logging
        //     }
        //   }
        // }

        var basePath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
        config.SetBasePath(basePath!);
        config.AddJsonFile("appsettings.json", optional: false);
        config.AddEnvironmentVariables();
    })
    .ConfigureServices(services =>
    {
        services.AddApplicationInsightsTelemetryWorkerService();
        services.ConfigureFunctionsApplicationInsights();
        services.Configure<LoggerFilterOptions>(options => {
          // The Application Insights SDK adds a default logging filter that instructs ILogger to capture only Warning and more severe logs. Application Insights requires an explicit override.
          // Log levels can also be configured using appsettings.json. For more information, see https://learn.microsoft.com/en-us/azure/azure-monitor/app/worker-service#ilogger-logs
          LoggerFilterRule? toRemove = options.Rules.FirstOrDefault(rule => rule.ProviderName
            == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");
          if (toRemove is not null)
          {
              options.Rules.Remove(toRemove);
          }
      });

        services.ConfigureStartupServices(); // bring up your own Startup class to register DI services
    })
    .ConfigureLogging((hostingContext, logging) =>
    {
        var services = logging.Services;
        logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
    })
    .Build();

host.Run();

[piotrzbyszynski]

@springcomp Is it expected to work also on .NET 9 in Isolated Model? It seems that I am not able to control log levels using apsettings.json. Also, no matter settings in host.json and local.settings.json AI logs all log levels. I can not restrict AI to log only specific levels or categories.

[LockTar]

A few years back I started this thread. It's amazing that a product like this is changing this much but simple logging is still so difficult. Last November, I created a new version of my sample app just to get logging working. I know it's .NET 8 and I think a few packages are not up to date anymore but it works.

Logging lower than information is still a problem but I can live without it. The most important thing is that I can log every function call. I can control it how I like it.

Another important thing is that it's supported by Microsoft. No custom classes, no difficult setup. Just following all the cryptic pages of documentation of Microsoft. And that's the real problem, the documentation is to wide for just setting up logging in Azure Functions (and of course the missing features).

Creating a new project template of my project would help... Last time that I did this was 10 years back. Anyone with more recent knowledge with this? I would love to have a PR.

[springcomp]

@springcomp Is it expected to work also on .NET 9 in Isolated Model? It seems that I am not able to control log levels using apsettings.json. Also, no matter settings in host.json and local.settings.json AI logs all log levels. I can not restrict AI to log only specific levels or categories.

Yes indeed, this works in .NET 9.

I have contributed a consolidated lesson at Azure Functions University. That is fundamentally a rehash of @LockTar ’s solution that everyone rediscovers by putting all the pieces of the puzzle in the right place.

Please, use comments in that pull request to highlight your issues ; I would be happy to take some time to guide you.

But it all boils down to this:

• Enabling Application Insights
• Removing the known limitation that prevent logging levels lower than LogLevel.Warning
• Making sure appsettings.json is taken into account by making its path explicit under Linux.
• Making sure you do not use . (period) characters in your category names if you want to override log levels using App Settings and environment variables.

Please, feel free to chime in in the commnets of the referred to pull request.

[bc3tech]

From what I can tell, yes, this lights up logs in AI as we want however it does not enable structure logging which is pretty critical.

For example, this log line:

logger.LogInformation("[{CustomOperationId}] Analysis received. {NumLines} line(s) in result.", message.GetProperty("OperationId").GetString(), message.GetProperty("Text").GetString().Split('\n').Length)

SHOULD result in an entry in AI with customDimensions having two additional properties under it, namely CustomOperationId and NumLines but also prop_originalFormat. However, AI doesn't have any of these:

[springcomp]

@bc3tech

I can attest that @LockTar's recently updated sample application produces the expected result in the CustomDimensions structure on Application Insights.

I can also attest that the consolidated sample i linked to in my previous comment does indeed produce the desired structured logging properties in the CustomDimensions structure on Application Insights.

Please, be aware of a few changes, though:

• The original Category custom property has now been renamed to CategoryName instead.
• The prop__ prefix is no longer appended once logs go directly from your Worker to Application Insights.
• Thus, using your example, you should have the following properties:
  • OriginalFormat
  • CustomOperationId
  • NumLines

In your example, you do not even have a custom CategoryName property. This leads me to suspect that you may be using an outdated version of the Microsoft.Azure.Functions.Worker.ApplicationInsights NuGet package. Recent versions of this package will change the way logs are sent to Application Insights. Instead of being sent back to the Functions Runtime host and relayed to Application Insights like in the past, they will now be emitted by your worker and sent directly to Application Insights.