report manager Documentation

report-manager

This program utilizes the reports API to maintain a file system tree with the option to send notifications when new reports are downloaded for processing.

Queue options:

• Apache Kafka
• Redis
• STOMP (ActiveMQ, RabbitMQ)

Usage: report-manager.py /path/to/config.ini [ days ]

The optional 'days' argument is the number of previous days to download reports for. The default is 2.

Sample configuration file:

[reports]
directory = /var/tmp/reports
min_disk_free = 512
notifier = none
url_prefix = http://myserver/reports/

[stomp]
server = 127.0.0.1
port = 61613
user = guest
password = guest
queue = /queue/mytest

[redis]
server = 127.0.0.1
port = 6379
;password = guest
queue = mytest

[kafka]
server = 127.0.0.1
port = 9092
queue = mytest

The [reports] section is required and must contain the 'directory' key.

Settings:

• directory : top level directory to store reports in
• min_disk_free : minimum disk free in MB to attempt download (default 512)
• notifier : specify a notification queue type [none, stomp, redis, or kafka]
• url_prefix : URL prefix replacement for the top level directory for notification messages

If a 'notifier' is configured in the [reports] section, an additional section with a matching name is required.

Settings:

• server : server IP address or host name
• port : server port
• queue : queue identifier
• user : user name (if required)
• password : password (if required)

The notification entry is a JSON object that contains a timestamp, report date, report type, and uri:

{
   "timestamp" : "2022-09-01 11:32:45",
   "report_date" : "2022-08-31",
   "report_type" : "scan_stun",
   "uri" : "http://myserver/reports/2022/08/31/2022-08-31-scan_stun_example_com-asn.csv"
}

Example crontab to check for new downloads once per hour:

15 * * * * /opt/shadowserver/report-manager.py /opt/shadowserver/reports.ini