Skip to content

Get a detailed, real-time view of your PostgreSQL database and system metrics

License

Notifications You must be signed in to change notification settings

zalando/pg_view

Folders and files

NameName
Last commit message
Last commit date

Latest commit

949e3a9 · Feb 10, 2016
Jul 19, 2013
Jun 24, 2015
Jun 24, 2015
Jun 24, 2015
Feb 10, 2016
Jul 23, 2013
Jun 24, 2015
Jan 4, 2016
Feb 10, 2016
Jun 26, 2015
Jun 24, 2015
Feb 10, 2016
Jun 24, 2015

Repository files navigation

pg_view

Build Status PyPI Downloads License

PostgreSQL Activity View Utility

Synopsis

pg_view is a command-line tool to display the state to the PostgreSQL processes. It shows the per-process statistics combined with pg_stat_activity output for the processes that have the rows there, global system stats, per-partition information and the memory stats. You can find a blog post about it at tech.zalando.com.

Requirements

Linux 2.6, python 2.6, psycopg2, curses

By default, pg_view assumes it's able to connect to the local PostgreSQL instance with the user postgres and no password. On some systems it might be necessary to change your pg_hba.conf or set the password in .pgpass. A different user name can be specified in the configuration file, although specifying that file (with -c) turns off autodetection of connection parameters and available databases.

How it works:

The program queries system /process information files once every tick (by default the tick is 1s). It also runs some external programs, like df or du to get filesystem information. The latter might put an extra load on a disk subsystem.

Screenshot

pg_view screenshot

Connection arguments

By default, pg_view tries to autodetect all PostgreSQL clusters running on the host it's running at. To achieve this it performs the following steps:

  • read /proc/ filesystem and detect pid files for the postmaster processes
  • get the working directories from the symlink at /proc/pid/cwd
  • get to the working directories and read PG_VERSION for PostgreSQL verions. If we can't, assume it's not a PostgreSQL directory and skip.
  • try to get all sockets the process is listening to from /proc/net/unix, /proc/net/tcp and /proc/net/tcp6
  • if that fails and version is 9.1 or above, read connection arguments from postmaster.pid
  • check all arguments, picking the first one that allows us to establish a connection
  • if we can't get either the port/host or port/socket_directory pair, bail out.

If the program is unable to detect connection arguments using the algorithm above it's possible to specify those arguments manually using the configuration file supplied with -c option. This file should consist of one or more sections, containing key = value pairs. Each section's title represents a database cluster name, this name is only used to for display purposes (the actual name of the DB to connect to can be specified by the dbname parameter and is 'postgres' by default), and the key - value pairs should contain connection parameters. The valid keys are:

host
hostname or ip address, or unix_socket_directory path of the database server
port
the port the database server listsens on
user
database role name

The special 'DEFAULT' contains the parameters that apply for every database cluster if the corresponding parameter is missing from the database-specific section. For instance:

[DEFAULT]
port=5435

[testdb]
host=localhost

[testdb2]
host=/tmp/test

[testdb3]
host=192.168.1.0
port=5433
dbname=test

The application will try to connect to both testdb and testdb2 clusters using port 5435 (database postgres) upon reading this file, while testdb3 will be reached using port 5433 and database name 'test'.

Finally, if the auto-detection code works for you, it's possible to select only a single database by specifying the database instance name (in most cases mathes the last component of $PGDATA) with -i command-line option. If there are more thana single instance with the same name - you can additionally specify the required PG version with -V.

Usage

see python pg_view --help

Output: The tool supports 3 output methods: * ncurses (default) * console (-o console) * json (-o json).

Below is the description of some of the options:

  • system
    • iowait
      the percent of the CPU resources waiting on I/O
    • ctxt
      the number of context switches in the system
    • run, block
      the number of running and waiting processes.
    • For other parameters, please, refer to man 5 proc and look for /proc/stat
  • memory
    • dirty

      the total amount of memory waiting to be written on disk. The higher the value is, the more one has to wait during the flush.

    • as

      (CommittedAs) the total amount of memory required to store the workload in the worst case scenario.

    • limit

      maximum amount of memory that can be physically allocated. If as is higher than the limit - the processes will start getting out of memory errors, which will lead to PostgreSQL shutdown (but not to the data corruption.

      For the explanation of other parameters, please, refer to the Linux kernel documentation

  • partitions
    • type
      either containing database data (data) or WAL (xlog)
    • fill
      the rate of adding new data to the corresponding directory (/data or /pg_xlog).
    • until_full
      the time until the current partition will run out of space if we only consider writes to the corresponding data directory (/data or /pg_xlog). This column is only shown during the warning (3h) or critical (1h) conditions. This column only considers momentary writes, so if a single process writes 100MB/s on a partition with remaining 100GB left for only 2 seconds, it will show a critial status during those 2 seconds.
    • total, left, read, write
      the amount of space total, free, read and write rate (MB/s) on a partition. Note that write rate is different from fill rate: it considers the whole partition, not only Postgres directories and shows data modifications, i.e deletion of files at the rate of 10MB/s will be shown as a positive write rate.
    • path_size
      size of the corresponding PostgreSQL directory.
  • postgres processes
    • type
      either a system process (autovacuum launcher, logger, archiver, etc) or a process that executes queries (backend or autovacuum). By default, only user processes are shown (press 's' to show all of them) in curses mode, and all in the console one.
    • s
      process state (R - 'running', S - 'sleeping', D - 'uninterruptable sleep', see man ps for more details).
    • utime, stime, guest
      consumption of CPU resources by process. Since PostgreSQL backends can't use more than one CPU, the percentage of a single CPU time is shown here.
    • read, write
      amount of data read or written from the partition (in MB/s).
    • age
      time from the process start
    • db
      the database the process runs on
    • query
      the query the process executes.

Hotkeys:

  • f
    instantly freeze the output. Press f for the second time to resume.
  • u
    toggle display of measurement units.
  • a
    auto-hide some of the fields from the PostgreSQL output. Currently, if this option is turned to on, the following fields can be hidden to leave space for the remaining ones: type, s, utime, stime, guest
  • h
    show the help screen

Releasing

$ ./release.sh <NEW-VERSION>

License

Apache 2.0