How to filter out extra log categories

[zvrba]

Situation: Azure Function, ServiceBus trigger, "standard" AF setup for NET8 isolated. We have a Serilog logging back-end that sends data to Seq.

No matter what I do from the solutions mentioned in the issues below, the logs keep appearing. I've spent a whole workday on this, and nothing seems to work. Is there a reliable step-by-step recipe, either through configuration or code, to get rid of these logs, without setting the global level to "Warning"?

Related links #1025 #2531 #1182 Azure/azure-functions-host#9259 Azure/azure-functions-core-tools#3059

[jviau]

Those logs are from HttpClient, which are not owned by the functions team. You can control them via the category "System.Net.Http.HttpClient". Since we have two processes in play for dotnet isolated, you may need to set them separately: host is via host.json. The worker process is in your control and depends on how you ingest configuration.

Also, please refer to https://opensource.microsoft.com/codeofconduct/ for future engagements.

[dwmorton18]

I ran into the same issue (and frustration), with other solutions such as setting the logLevel in host.json not working for me. I was finally able to suppress the unwanted HTTP Information logs, while showing my own custom Information logs, using this code snippet in my Program.cs. I don't know why I couldn't get the filter to work coming from host.json, but this worked for me.

.ConfigureLogging(logging =>
{
    // Disable default App Insights logging rule.
    logging.Services.Configure<LoggerFilterOptions>(options =>
    {
        LoggerFilterRule defaultRule = options.Rules.FirstOrDefault(rule => rule.ProviderName
            == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");
        if (defaultRule is not null)
        {
            options.Rules.Remove(defaultRule);
        }
    });

    // Suppress HTTP logs
    logging.AddFilter("System.Net.Http.HttpClient", LogLevel.Error);

})

[zvrba]

@jviau I guess we're talking about the worker process because the URI is due to settlement from ServiceBus trigger. The summary is at the bottom.

I attempted to put (as per various other issues)

  "logging": {
    "logLevel": {
      "default": "Information",
      "System.Net.Http.HttpClient": "Warning",
    },

into host.json, and was then greeted with the error below in the Azure portal. I don't have the full screenshot, but the exception complained about "." in property name.

Current content of host.json is

  "logging": {
    "logLevel": {
      "default": "Information",
      "Azure": "Warning",
      "System": "Warning",
      "Microsot": "Warning"
    },

without effect. My startup code is as follows

        var host = new HostBuilder()
            .ConfigureAppConfiguration(b =>
            {
                b.AddJsonFile("appsettings.json", true);
                b.AddAzureAppConfiguration(o => o.Connect(
                    MachineAmbient.GetAzureAppConfigUri(),
                    MachineAmbient.ServiceCredential));
            })
            .ConfigureLogging(b =>
            {
                b.AddFilter("System.Net.Http.HttpClient", LogLevel.Warning);
                b.AddFilter("System.Net.Http.HttpClient.CallInvokerExtractor.ClientHandler", LogLevel.Warning);
                b.AddFilter("System.Net.Http.HttpClient.CallInvokerExtractor.LogicalHandler", LogLevel.Warning);
                b.Services.Configure<LoggerFilterOptions>(options =>
                {
                    var defaultRule = options.Rules.FirstOrDefault(rule => rule.ProviderName
                        == "Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider");
                    if (defaultRule is not null)
                    {
                        options.Rules.Remove(defaultRule);
                    }
                });
            })
            .ConfigureFunctionsWebApplication()
            .ConfigureServices((hc, services) =>
            {
                services.AddApplicationInsightsTelemetryWorkerService();
                services.ConfigureFunctionsApplicationInsights();
                // ...
            });

appsettings.json contains

  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Azure": "Warning",
      "System": "Warning",
      "Microsot": "Warning"
    }
  },

I even added additional categories because a sample event in Seq looks as follows

In one of the issues, someone suggested that the category might be wrong and I went to check the complete log event in Azure portal, under "Logs" (I find them in "Dependencies" table), but the category is nowhere to be found??

So, in summary:

• We have a system component (System.Net.Http.HttpClient ?) that generates unwanted logs, presumably through Microsoft.Extensions.Logging
• Not even hard-coding the configuration as above (literal copy-paste from the deployed code) is able to remove the unwanted logs
• I'm unable to find the proper category. The above strings in the startup code is just guesswork.
• The above configuration removes the unwanted logs from console logs when running from Visual Studio (so the categories might be correct?), but not from other logs (Serilog, AppInsights).

Which gives?

Are filter rules evaluated before being given to the logging provider (in this case, Serilog with Seq sink, among others) or by the logging provider (i.e., I'm looking at the wrong place and I should be configuring Serilog provider instead)?

[zvrba]

I think I have found the culprit. Not yet tested, but I'm pretty sure that's the cause. As per https://learn.microsoft.com/en-us/dotnet/core/extensions/logging?tabs=command-line#how-filtering-rules-are-applied , rules for the specific provider take priority over everything else. Serilog's extension method on ILoggingBuilder contains the following line:

        builder.AddFilter<SerilogLoggerProvider>(null, LogLevel.Trace);

So no matter what I do with the defaults, the above rule will trump them.

EDIT: Finally resolved:

                services.RemoveAll<IHttpMessageHandlerBuilderFilter>();

[jviau]

into host.json, and was then greeted with the error below in the Azure portal. I don't have the full screenshot, but the exception complained about "." in property name.

I believe it is actually an issue with your trailing comma. We will be relaxing the parsing of host.json in the worker to avoid this in the future.

[zvrba]

The above was just an excerpt; there are no issues in the complete host.json file (Visual Studio doesn't complain -- no green squiggles). Sorry for the confusion.