Removes duplicate punctuation and whitespace from stdin. It will not touch alphanumeric chracters but if anything else appears more than once in a succession, they will be reduced to a single occurrence.
Syntax: undupe
There are no options or arguments
A before example: output from an OpenStack CLI that formats a table column but it can be rather difficult to read, especially on a screen without a wide width:
$ nova list
+--------------------------------------+---------------------------------------+--------+------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------------------------------------+--------+------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------+
| fd1afc31-104b-41d9-8978-30122c9ddb97 | autoit-f-autovnf-jpfuntne-avf-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.7; autoit-jcp_mgmt=40.101.13.9 |
| 1dd45d82-c774-497f-8211-14c2301f161f | autoit-f-autovnf-jpfuntne-avf-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.9; autoit-jcp_mgmt=40.101.13.5 |
| fcdd375f-37e1-4885-8c0f-d4aadad64e6e | jcp_autovnf-em-jpfuntne-em-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.10; autoit-jcp_mgmt=40.101.13.8 |
| ea9571a5-0430-4ad7-a2b3-7e50deabe5ba | jcp_autovnf-em-jpfuntne-em-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.14; autoit-jcp_mgmt=40.101.13.14 |
| 448ef86b-0bac-4de1-9fd2-a2afb5d6aaa4 | jcp_autovnf-esc-jpfuntne-esc-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.11; autoit-jcp_mgmt=40.101.13.18 |
| da2f282d-5088-4ac1-858a-9c9148eb01bc | jcp_autovnf-esc-jpfuntne-esc-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.17; autoit-jcp_mgmt=40.101.13.13 |
| 65bcf434-476c-4886-aa5d-37546aa4dda4 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-0 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.5; autoit-jcp_mgmt=40.101.13.15; di-internal1=192.168.10.200; di-internal2=192.168.11.49 |
| f112de69-86a4-4a80-be62-88fb0f010a82 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.6; autoit-jcp_mgmt=40.101.13.21; di-internal1=192.168.10.197; di-internal2=192.168.11.55 |
| 76388909-7c46-4c01-b3bc-862a034f0278 | jcp_autovnf-vPC-DI-PGW-jpfuntne-sf1-0 | ACTIVE | - | Running | service2=192.168.13.32; service1=192.168.12.109; autoit-jcp_orch=40.101.14.20; di-internal2=192.168.11.53; di-internal1=192.168.10.235 |
| d308c878-2c51-47a5-9a6d-23185983c9b6 | jpfuntne-autodeploy-6-3-1 | ACTIVE | - | Running | management=192.168.100.42, 10.225.202.207 |
| 36a7f0f4-55b3-4524-b060-5f86892a84d4 | jpfuntne-autoit-6-3-1 | ACTIVE | - | Running | management=192.168.100.41, 10.225.202.240 |
+--------------------------------------+---------------------------------------+--------+------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------+
$
An after example using the same command with this script used as a filter:
$ nova list | undupe
+-+-+-+-+-+-+
| ID | Name | Status | Task State | Power State | Networks |
+-+-+-+-+-+-+
| fd1afc31-104b-41d9-8978-30122c9ddb97 | autoit-f-autovnf-jpfuntne-avf-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.7; autoit-jcp_mgmt=40.101.13.9 |
| 1dd45d82-c774-497f-8211-14c2301f161f | autoit-f-autovnf-jpfuntne-avf-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.9; autoit-jcp_mgmt=40.101.13.5 |
| fcdd375f-37e1-4885-8c0f-d4aadad64e6e | jcp_autovnf-em-jpfuntne-em-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.10; autoit-jcp_mgmt=40.101.13.8 |
| ea9571a5-0430-4ad7-a2b3-7e50deabe5ba | jcp_autovnf-em-jpfuntne-em-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.14; autoit-jcp_mgmt=40.101.13.14 |
| 448ef86b-0bac-4de1-9fd2-a2afb5d6aaa4 | jcp_autovnf-esc-jpfuntne-esc-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.11; autoit-jcp_mgmt=40.101.13.18 |
| da2f282d-5088-4ac1-858a-9c9148eb01bc | jcp_autovnf-esc-jpfuntne-esc-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.17; autoit-jcp_mgmt=40.101.13.13 |
| 65bcf434-476c-4886-aa5d-37546aa4dda4 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-0 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.5; autoit-jcp_mgmt=40.101.13.15; di-internal1=192.168.10.200; di-internal2=192.168.11.49 |
| f112de69-86a4-4a80-be62-88fb0f010a82 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.6; autoit-jcp_mgmt=40.101.13.21; di-internal1=192.168.10.197; di-internal2=192.168.11.55 |
| 76388909-7c46-4c01-b3bc-862a034f0278 | jcp_autovnf-vPC-DI-PGW-jpfuntne-sf1-0 | ACTIVE | - | Running | service2=192.168.13.32; service1=192.168.12.109; autoit-jcp_orch=40.101.14.20; di-internal2=192.168.11.53; di-internal1=192.168.10.235 |
| d308c878-2c51-47a5-9a6d-23185983c9b6 | jpfuntne-autodeploy-6-3-1 | ACTIVE | - | Running | management=192.168.100.42, 10.225.202.207 |
| 36a7f0f4-55b3-4524-b060-5f86892a84d4 | jpfuntne-autoit-6-3-1 | ACTIVE | - | Running | management=192.168.100.41, 10.225.202.240 |
+-+-+-+-+-+-+
$
This is a little more managable and easier to read. Sure, there are advantages to the table too but if you grep out the lines with very long data that's making the table so wide to begin with, the remaining rows in the original output still have a lot of useless and unnecessary padding. This script addresses that.
-
The script only reads from stdin.
-
White space (except for newlines) are treated the same. So if you have
' \t '
(space, tab, space), that will be replaced with a single space. This is the only exception - each punctuation character is treated sepearately. -
The use of the
fitwidth
script might be be good after using this script. This script would reformat lines that don't need to be super long andfitwidth
could reduce the lines long lines:$ nova list | undupe | fitwidth --width 180 +-+-+-+-+-+-+ | ID | Name | Status | Task State | Power State | Networks | +-+-+-+-+-+-+ | fd1afc31-104b-41d9-8978-30122c9ddb97 | autoit-f-autovnf-jpfuntne-avf-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.7; autoit-jcp_mgmt=40.101.13.9 | | 1dd45d82-c774-497f-8211-14c2301f161f | autoit-f-autovnf-jpfuntne-avf-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.9; autoit-jcp_mgmt=40.101.13.5 | | fcdd375f-37e1-4885-8c0f-d4aadad64e6e | jcp_autovnf-em-jpfuntne-em-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.10; autoit-jcp_mgmt=40.101.13.8 | | ea9571a5-0430-4ad7-a2b3-7e50deabe5ba | jcp_autovnf-em-jpfuntne-em-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.14; autoit-jcp_mgmt=40.101.13.14 | | 448ef86b-0bac-4de1-9fd2-a2afb5d6aaa4 | jcp_autovnf-esc-jpfuntne-esc-1 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.11; autoit-jcp_mgmt=40.101.13.18 | | da2f282d-5088-4ac1-858a-9c9148eb01bc | jcp_autovnf-esc-jpfuntne-esc-2 | ACTIVE | - | Running | autoit-jcp_orch=40.101.14.17; autoit-jcp_mgmt=40.101.13.13 | | 65bcf434-476c-4886-aa5d-37546aa4dda4 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-0 | ACTIVE ... autoit-jcp_mgmt=40.101.13.15; di-internal1=192.168.10.200; di-internal2=192.168.11.49 | | f112de69-86a4-4a80-be62-88fb0f010a82 | jcp_autovnf-vPC-DI-PGW-jpfuntne-cf1-1 | ACTIVE ... autoit-jcp_mgmt=40.101.13.21; di-internal1=192.168.10.197; di-internal2=192.168.11.55 | | 76388909-7c46-4c01-b3bc-862a034f0278 | jcp_autovnf-vPC-DI-PGW-jpfuntne-sf1-0 | ACTIVE ... autoit-jcp_orch=40.101.14.20; di-internal2=192.168.11.53; di-internal1=192.168.10.235 | | d308c878-2c51-47a5-9a6d-23185983c9b6 | jpfuntne-autodeploy-6-3-1 | ACTIVE | - | Running | management=192.168.100.42, 10.225.202.207 | | 36a7f0f4-55b3-4524-b060-5f86892a84d4 | jpfuntne-autoit-6-3-1 | ACTIVE | - | Running | management=192.168.100.41, 10.225.202.240 | +-+-+-+-+-+-+
As of 2019/01/07, I still hadn't gotten around to writing doc for
fitwidth
but the script is in the repo and I'll try to do get to the doc soon.