Levels of diagnostic logs in AKS

[Gordonby]

My deployment is failing on this rule.

Rule: Azure.AKS.PlatformLogs
Reason: The diagnostic setting (aksDiags) logs should enable (cluster-autoscaler, kube-apiserver, kube-controller-manager, kube-scheduler, AllMetrics).

However the AKS Secure Baseline Cluster Stamp (which is generally the go-to reference implementation), only configures 4
cluster-autoscaler, kube-controller-manager, kube-audit-admin, guard.

ref: https://github.com/mspnp/aks-secure-baseline/blob/ed5206c3b1cb620a089201951290580857961ea4/cluster-stamp.json#L1227

[BernieWhite]

@Gordonby That is fair. The implementation was based on https://docs.microsoft.com/en-gb/azure/aks/monitor-aks#collect-resource-logs which does suggest AllMetrics should be logged but I can see why that may not be desired.

Does it help to know that you can override the default configuration with the following to remove AllMetrics?

# YAML: The default AZURE_AKS_ENABLED_PLATFORM_LOG_CATEGORIES_LIST configuration option
configuration:
  AZURE_AKS_ENABLED_PLATFORM_LOG_CATEGORIES_LIST: ['cluster-autoscaler', 'kube-apiserver', 'kube-controller-manager', 'kube-scheduler']

@ArmaanMcleod Thoughts?

[ArmaanMcleod]

@Gordonby @BernieWhite From a Secure baseline perspective, I can see why only cluster-autoscaler, kube-controller-manager, kube-audit-admin, guard would only be enabled.

Currently the AKS logs are validated like so:

• Azure.Aks.AuditLogs -> enable at least one of (kube-audit, kube-audit-admin) and guard.

• Azure.Aks.PlatformLogs -> enable cluster-autoscaler, kube-apiserver, kube-controller-manager, kube-scheduler, AllMetrics

Can remove AllMetrics from the default for Azure.Aks.PlatformLogs if you guys want, but this can easily be overriden with the AZURE_AKS_ENABLED_PLATFORM_LOG_CATEGORIES_LIST option as shown above. I think we kept it in there since it made sense to have all platform metrics being sent to Log Analytics by default. Happy to change this though.

[BernieWhite]

@Gordonby Did this answer your question?

[Gordonby]

indeed 😸