Skip to content

Latest commit

 

History

History
130 lines (94 loc) · 6.68 KB

kubernetes-volumes.md

File metadata and controls

130 lines (94 loc) · 6.68 KB
Raw

kubernetes-volumes

Monitor Type: kubernetes-volumes (Source)

Accepts Endpoints: No

Multiple Instances Allowed: Yes

Overview

This monitor sends usage stats about volumes mounted to Kubernetes pods (e.g. free space/inodes). This information is gotten from the Kubelet /stats/summary endpoint. The normal filesystems monitor generally will not report Persistent Volume usage metrics because those volumes are not seen by the agent since they can be mounted dynamically and older versions of K8s don't support mount propagation of those mounts to the agent container.

Dimensions that identify the underlying volume source will be added for awsElasticBlockStore, gcePersistentDisk and glusterfs persistent volumes, and for configMap, downwardAPI, emptyDir and secret non-persistent volumes. Support for more can be easily added as needed.

If interested in collecting metrics from Persistent Volumes and Persistent Volume Claims from a RBAC enabled cluster the following permissions need to be granted to the Agent.

- apiGroups:
    - ""
  resources:
    - persistentvolumes
    - persistentvolumeclaims
  verbs:
    - get

Configuration

To activate this monitor in the Smart Agent, add the following to your agent config:

monitors:  # All monitor config goes under this key
 - type: kubernetes-volumes
   ...  # Additional config

For a list of monitor options that are common to all monitors, see Common Configuration.

Config option Required Type Description
kubeletAPI no object (see below) Kubelet kubeletClient configuration
kubernetesAPI no object (see below) Configuration of the Kubernetes API kubeletClient

The nested kubeletAPI config object has the following fields:

Config option Required Type Description
url no string URL of the Kubelet instance. This will default to http://<current node hostname>:10255 if not provided.
authType no string Can be none for no auth, tls for TLS client cert auth, or serviceAccount to use the pod's default service account token to authenticate. (default: none)
skipVerify no bool Whether to skip verification of the Kubelet's TLS cert (default: true)
caCertPath no string Path to the CA cert that has signed the Kubelet's TLS cert, unnecessary if skipVerify is set to false.
clientCertPath no string Path to the client TLS cert to use if authType is set to tls
clientKeyPath no string Path to the client TLS key to use if authType is set to tls
logResponses no bool Whether to log the raw cadvisor response at the debug level for debugging purposes. (default: false)

The nested kubernetesAPI config object has the following fields:

Config option Required Type Description
authType no string How to authenticate to the K8s API server. This can be one of none (for no auth), tls (to use manually specified TLS client certs, not recommended), serviceAccount (to use the standard service account token provided to the agent pod), or kubeConfig to use credentials from ~/.kube/config. (default: serviceAccount)
skipVerify no bool Whether to skip verifying the TLS cert from the API server. Almost never needed. (default: false)
clientCertPath no string The path to the TLS client cert on the pod's filesystem, if using tls auth.
clientKeyPath no string The path to the TLS client key on the pod's filesystem, if using tls auth.
caCertPath no string Path to a CA certificate to use when verifying the API server's TLS cert. Generally this is provided by K8s alongside the service account token, which will be picked up automatically, so this should rarely be necessary to specify.

Metrics

These are the metrics available for this monitor. Metrics that are categorized as container/host (default) are in bold and italics in the list below.

  • kubernetes.volume_available_bytes (gauge)
    The number of available bytes in the volume
  • kubernetes.volume_capacity_bytes (gauge)
    The total capacity in bytes of the volume
  • kubernetes.volume_inodes (gauge)
    The total inodes in the filesystem
  • kubernetes.volume_inodes_free (gauge)
    The free inodes in the filesystem
  • kubernetes.volume_inodes_used (gauge)
    The inodes used by the filesystem. This may not equal inodes - free because filesystem may share inodes with other filesystems.

Non-default metrics (version 4.7.0+)

To emit metrics that are not default, you can add those metrics in the generic monitor-level extraMetrics config option. Metrics that are derived from specific configuration options that do not appear in the above list of metrics do not need to be added to extraMetrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.

Dimensions

The following dimensions may occur on metrics emitted by this monitor. Some dimensions may be specific to certain metrics.

Name Description
VolumeId (EBS volumes only) The EBS volume id of the underlying volume source
endpoints_name (GlusterFS volumes only) The endpoint name used for the GlusterFS volume
fs_type (EBS volumes and GCE persistent disks only) The filesystem type of the underlying EBS volume or GCE persistent disk
glusterfs_path (GlusterFS volumes only) The GlusterFS volume path
kubernetes_namespace The namespace of the pod that has this volume
kubernetes_pod_name The name of the pod that has this volume
kubernetes_pod_uid The UID of the pod that has this volume
partition (EBS volumes and GCE persistent disks only) The partition number of the underlying EBS volume or GCE persistent disk (0 indicates the entire disk)
pd_name (GCE persistent disks only) The GCE persistent disk name of the underlying volume source
volume The volume name as given in the pod spec under volumes
volume_type The type of the underlying volume -- this will be the key used in the k8s volume config spec (e.g. awsElasticBlockStore, gcePersistentDisk, configMap, secret, etc.)