Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

state tag set under zpool_stats should be a field set instead #9 #12

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

state tag set under zpool_stats should be a field set instead #9 #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cb0c3b3
state tag set under zpool_stats should be a field set instead #9
richardelling Feb 25, 2023
7b3f38e
quotes added to string field
richardelling Feb 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
200 changes: 101 additions & 99 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ zpool_influxdb [options] [poolname]
```
If no poolname is specified, then all pools are sampled.

| option | short option | description |
|---|---|---|
| --execd | -e | For use with telegraf's `execd` plugin. When [enter] is pressed, the pools are sampled. To exit, use [ctrl+D] |
| --no-histogram | -n | Do not print histogram information |
| --sum-histogram-buckets | -s | Sum histogram bucket values |
| --help | -h | Print a short usage message |
| option | short option | description |
|-------------------------|--------------|---------------------------------------------------------------------------------------------------------------|
| --execd | -e | For use with telegraf's `execd` plugin. When [enter] is pressed, the pools are sampled. To exit, use [ctrl+D] |
| --no-histogram | -n | Do not print histogram information |
| --sum-histogram-buckets | -s | Sum histogram bucket values |
| --help | -h | Print a short usage message |

#### Histogram Bucket Values
The histogram data collected by ZFS is stored as independent bucket values.
Expand All @@ -49,39 +49,40 @@ as summed values.
## Measurements
The following measurements are collected:

| measurement | description | zpool equivalent |
|---|---|---|
| zpool_stats | general size and data | zpool list |
| zpool_scan_stats | scrub, rebuild, and resilver statistics (omitted if no scan has been requested) | zpool status |
| zpool_vdev_stats | per-vdev statistics | zpool iostat -q |
| zpool_io_size | per-vdev I/O size histogram | zpool iostat -r |
| zpool_latency | per-vdev I/O latency histogram | zpool iostat -w |
| zpool_vdev_queue | per-vdev instantaneous queue depth | zpool iostat -q |
| measurement | description | zpool equivalent |
|------------------|---------------------------------------------------------------------------------|------------------|
| zpool_stats | general size and data | zpool list |
| zpool_scan_stats | scrub, rebuild, and resilver statistics (omitted if no scan has been requested) | zpool status |
| zpool_vdev_stats | per-vdev statistics | zpool iostat -q |
| zpool_io_size | per-vdev I/O size histogram | zpool iostat -r |
| zpool_latency | per-vdev I/O latency histogram | zpool iostat -w |
| zpool_vdev_queue | per-vdev instantaneous queue depth | zpool iostat -q |

### zpool_stats Description
zpool_stats contains top-level summary statistics for the pool.
Performance counters measure the I/Os to the pool's devices.

#### zpool_stats Tags

| label | description |
|---|---|
| name | pool name |
| label | description |
|-------|----------------------------------------|
| name | pool name |
| state | pool state, as shown by _zpool status_ |

#### zpool_stats Fields

| field | units | description |
|---|---|---|
| alloc | bytes | allocated space |
| free | bytes | unallocated space |
| size | bytes | total pool size |
| read_bytes | bytes | bytes read since pool import |
| read_errors | count | number of read errors |
| read_ops | count | number of read operations |
| write_bytes | bytes | bytes written since pool import |
| write_errors | count | number of write errors |
| write_ops | count | number of write operations |
| field | units | description |
|--------------|--------|---------------------------------|
| alloc | bytes | allocated space |
| free | bytes | unallocated space |
| size | bytes | total pool size |
| read_bytes | bytes | bytes read since pool import |
| read_errors | count | number of read errors |
| read_ops | count | number of read operations |
| vdev_state | string | same as tag "state" |
| write_bytes | bytes | bytes written since pool import |
| write_errors | count | number of write errors |
| write_ops | count | number of write operations |

### zpool_scan_stats Description
Once a pool has been scrubbed, resilvered, or rebuilt, the zpool_scan_stats
Expand All @@ -91,28 +92,29 @@ cannot be reported by this collector.

#### zpool_scan_stats Tags

| label | description |
|---|---|
| name | pool name |
| label | description |
|----------|---------------------------------------------------------|
| name | pool name |
| function | name of the scan function running or recently completed |
| state | scan state, as shown by _zpool status_ |
| state | scan state, as shown by _zpool status_ |

#### zpool_scan_stats Fields

| field | units | description |
|---|---|---|
| errors | count | number of errors encountered by scan |
| examined | bytes | total data examined during scan |
| to_examine | bytes | prediction of total bytes to be scanned |
| pass_examined | bytes | data examined during current scan pass |
| processed | bytes | data reconstructed during scan |
| to_process | bytes | total bytes to be repaired |
| rate | bytes/sec | examination rate |
| start_ts | epoch timestamp | start timestamp for scan |
| pause_ts | epoch timestamp | timestamp for a scan pause request |
| end_ts | epoch timestamp | completion timestamp for scan |
| paused_t | seconds | elapsed time while paused |
| remaining_t | seconds | estimate of time remaining for scan |
| field | units | description |
|---------------|-----------------|-----------------------------------------|
| errors | count | number of errors encountered by scan |
| examined | bytes | total data examined during scan |
| to_examine | bytes | prediction of total bytes to be scanned |
| pass_examined | bytes | data examined during current scan pass |
| processed | bytes | data reconstructed during scan |
| to_process | bytes | total bytes to be repaired |
| rate | bytes/sec | examination rate |
| start_ts | epoch timestamp | start timestamp for scan |
| pause_ts | epoch timestamp | timestamp for a scan pause request |
| end_ts | epoch timestamp | completion timestamp for scan |
| paused_t | seconds | elapsed time while paused |
| remaining_t | seconds | estimate of time remaining for scan |
| scan_state | string | same as tag "state" |

### zpool_vdev_stats Description
The ZFS I/O (ZIO) scheduler uses five queues to schedule I/Os to each vdev.
Expand All @@ -127,24 +129,24 @@ the sample time. Therefore, it is not unusual to see all zeroes
for an idle pool.

#### zpool_vdev_stats Tags
| label | description |
|---|---|
| name | pool name |
| vdev | vdev name (root = entire pool) |
| label | description |
|-------|--------------------------------|
| name | pool name |
| vdev | vdev name (root = entire pool) |

#### zpool_vdev_stats Fields
| field | units | description |
|---|---|---|
| sync_r_active_queue | entries | synchronous read active queue depth |
| sync_w_active_queue | entries | synchronous write active queue depth |
| async_r_active_queue | entries | asynchronous read active queue depth |
| async_w_active_queue | entries | asynchronous write active queue depth |
| async_scrub_active_queue | entries | asynchronous scrub active queue depth |
| sync_r_pend_queue | entries | synchronous read pending queue depth |
| sync_w_pend_queue | entries | synchronous write pending queue depth |
| async_r_pend_queue | entries | asynchronous read pending queue depth |
| async_w_pend_queue | entries | asynchronous write pending queue depth |
| async_scrub_pend_queue | entries | asynchronous scrub pending queue depth |
| field | units | description |
|--------------------------|---------|----------------------------------------|
| sync_r_active_queue | entries | synchronous read active queue depth |
| sync_w_active_queue | entries | synchronous write active queue depth |
| async_r_active_queue | entries | asynchronous read active queue depth |
| async_w_active_queue | entries | asynchronous write active queue depth |
| async_scrub_active_queue | entries | asynchronous scrub active queue depth |
| sync_r_pend_queue | entries | synchronous read pending queue depth |
| sync_w_pend_queue | entries | synchronous write pending queue depth |
| async_r_pend_queue | entries | asynchronous read pending queue depth |
| async_w_pend_queue | entries | asynchronous write pending queue depth |
| async_scrub_pend_queue | entries | asynchronous scrub pending queue depth |

### zpool_latency Histogram
ZFS tracks the latency of each I/O in the ZIO pipeline. This latency can
Expand All @@ -156,26 +158,26 @@ The largest bucket is tagged "le=+Inf", representing the total count
of I/Os by type and vdev.

#### zpool_latency Histogram Tags
| label | description |
|---|---|
| le | bucket for histogram, latency is less than or equal to bucket value in seconds |
| name | pool name |
| path | for leaf vdevs, the device path name, otherwise omitted |
| vdev | vdev name (root = entire pool) |
| label | description |
|-------|--------------------------------------------------------------------------------|
| le | bucket for histogram, latency is less than or equal to bucket value in seconds |
| name | pool name |
| path | for leaf vdevs, the device path name, otherwise omitted |
| vdev | vdev name (root = entire pool) |

#### zpool_latency Histogram Fields
| field | units | description |
|---|---|---|
| total_read | operations | read operations of all types |
| field | units | description |
|-------------|------------|-------------------------------|
| total_read | operations | read operations of all types |
| total_write | operations | write operations of all types |
| disk_read | operations | disk read operations |
| disk_write | operations | disk write operations |
| sync_read | operations | ZIO sync reads |
| sync_write | operations | ZIO sync writes |
| async_read | operations | ZIO async reads|
| async_write | operations | ZIO async writes |
| scrub | operations | ZIO scrub/scan reads |
| trim | operations | ZIO trim (aka unmap) writes |
| disk_read | operations | disk read operations |
| disk_write | operations | disk write operations |
| sync_read | operations | ZIO sync reads |
| sync_write | operations | ZIO sync writes |
| async_read | operations | ZIO async reads |
| async_write | operations | ZIO async writes |
| scrub | operations | ZIO scrub/scan reads |
| trim | operations | ZIO trim (aka unmap) writes |

### zpool_io_size Histogram
ZFS tracks I/O throughout the ZIO pipeline. The size of each I/O is used
Expand All @@ -196,28 +198,28 @@ Note: trim I/Os can be larger than 16MiB, but the larger sizes are
accounted in the 16MiB bucket.

#### zpool_io_size Histogram Tags
| label | description |
|---|---|
| le | bucket for histogram, I/O size is less than or equal to bucket value in bytes |
| name | pool name |
| path | for leaf vdevs, the device path name, otherwise omitted |
| vdev | vdev name (root = entire pool) |
| label | description |
|-------|-------------------------------------------------------------------------------|
| le | bucket for histogram, I/O size is less than or equal to bucket value in bytes |
| name | pool name |
| path | for leaf vdevs, the device path name, otherwise omitted |
| vdev | vdev name (root = entire pool) |

#### zpool_io_size Histogram Fields
| field | units | description |
|---|---|---|
| sync_read_ind | blocks | independent sync reads |
| sync_write_ind | blocks | independent sync writes |
| async_read_ind | blocks | independent async reads |
| async_write_ind | blocks | independent async writes |
| scrub_read_ind | blocks | independent scrub/scan reads |
| trim_write_ind | blocks | independent trim (aka unmap) writes |
| sync_read_agg | blocks | aggregated sync reads |
| sync_write_agg | blocks | aggregated sync writes |
| async_read_agg | blocks | aggregated async reads |
| async_write_agg | blocks | aggregated async writes |
| scrub_read_agg | blocks | aggregated scrub/scan reads |
| trim_write_agg | blocks | aggregated trim (aka unmap) writes |
| field | units | description |
|-----------------|--------|-------------------------------------|
| sync_read_ind | blocks | independent sync reads |
| sync_write_ind | blocks | independent sync writes |
| async_read_ind | blocks | independent async reads |
| async_write_ind | blocks | independent async writes |
| scrub_read_ind | blocks | independent scrub/scan reads |
| trim_write_ind | blocks | independent trim (aka unmap) writes |
| sync_read_agg | blocks | aggregated sync reads |
| sync_write_agg | blocks | aggregated sync writes |
| async_read_agg | blocks | aggregated async reads |
| async_write_agg | blocks | aggregated async writes |
| scrub_read_agg | blocks | aggregated scrub/scan reads |
| trim_write_agg | blocks | aggregated trim (aka unmap) writes |

#### About unsigned integers
Telegraf v1.6.2 and later support unsigned 64-bit integers which more
Expand Down
15 changes: 10 additions & 5 deletions zpool_influxdb.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ print_scan_status(nvlist_t *nvroot, const char *pool_name) {
"pass_examined="IFMT",pause_ts="IFMT",paused_t="IFMT","
"pct_done=%.2f,processed="IFMT",rate="IFMT","
"remaining_t="IFMT",start_ts="IFMT","
"to_examine="IFMT",to_process="IFMT" ",
"to_examine="IFMT",to_process="IFMT",scan_state=\"%s\"",
MASK_UINT64(ps->pss_end_time),
MASK_UINT64(ps->pss_errors),
MASK_UINT64(examined),
Expand All @@ -255,9 +255,10 @@ print_scan_status(nvlist_t *nvroot, const char *pool_name) {
MASK_UINT64(remaining_time),
MASK_UINT64(ps->pss_start_time),
MASK_UINT64(ps->pss_to_examine),
MASK_UINT64(ps->pss_to_process)
MASK_UINT64(ps->pss_to_process),
state[ps->pss_state]
);
(void) printf("%lu\n", timestamp);
(void) printf(" %lu\n", timestamp);
return (0);
}

Expand Down Expand Up @@ -368,7 +369,8 @@ print_summary_stats(nvlist_t *nvroot, const char *pool_name,
(void) printf("alloc="IFMT",free="IFMT",size="IFMT","
"read_bytes="IFMT",read_errors="IFMT",read_ops="IFMT","
"write_bytes="IFMT",write_errors="IFMT",write_ops="IFMT","
"checksum_errors="IFMT",fragmentation="IFMT"",
"checksum_errors="IFMT",fragmentation="IFMT
",vdev_state=\"%s\"",
MASK_UINT64(vs->vs_alloc),
MASK_UINT64(vs->vs_space - vs->vs_alloc),
MASK_UINT64(vs->vs_space),
Expand All @@ -379,7 +381,10 @@ print_summary_stats(nvlist_t *nvroot, const char *pool_name,
MASK_UINT64(vs->vs_write_errors),
MASK_UINT64(vs->vs_ops[ZIO_TYPE_WRITE]),
MASK_UINT64(vs->vs_checksum_errors),
MASK_UINT64(vs->vs_fragmentation));
MASK_UINT64(vs->vs_fragmentation),
zpool_state_to_name((vdev_state_t) vs->vs_state,
(vdev_aux_t) vs->vs_aux)
);
(void) printf(" %lu\n", timestamp);
return (0);
}
Expand Down