Add a report of software upgrade logs

[johannaengland]

Closes #2457.

Sub-device id/name and sub-device serial number are still missing.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.21%. Comparing base (282694b) to head (a9f14f2).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2466   +/-   ##
=======================================
  Coverage   60.21%   60.21%           
=======================================
  Files         601      601           
  Lines       43981    43981           
=======================================
  Hits        26481    26481           
  Misses      17500    17500           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Test results

     12 files       12 suites   11m 34s ⏱️
3 318 tests 3 318 ✔️ 0 💤 0 ❌
9 429 runs  9 429 ✔️ 0 💤 0 ❌

Results for commit a9f14f2.

♻️ This comment has been updated with latest results.

[johannaengland]

Any pointers for the missing information?

[johannaengland]

The old software version is saved in the varmap. Is there any way to access that?

[lunkwill42]

This looks fine to me so far :)

The old software version is saved in the varmap. Is there any way to access that?

I can't recall what the attribute names are. Care to remind me? :)

[johannaengland]

This looks fine to me so far :)

The old software version is saved in the varmap. Is there any way to access that?

I can't recall what the attribute names are. Care to remind me? :)

https://github.com/johannaengland/nav/blob/master/python/nav/ipdevpoll/shadows/__init__.py#L304

[lunkwill42]

This looks fine to me so far :)

The old software version is saved in the varmap. Is there any way to access that?

I can't recall what the attribute names are. Care to remind me? :)

https://github.com/johannaengland/nav/blob/master/python/nav/ipdevpoll/shadows/__init__.py#L304

I checked this out, and, unfortunately (as I suspected): varmap items aren't automatically copied over from eventq to alerthist entries - which means that this information gets lost in the history log.

An eventengine plugin needs to explicitly copy over variables that we want to keep a historic record of. Otherwise, they're just around for the eventq/alertq for alert profile filter and alert message template expansion.

You can explicitly check the history of upgrade-type alert with something like this:

SELECT
    *
FROM
    alerthistvar
WHERE
    alerthistid IN (
        SELECT
            ah.alerthistid
        FROM
            alerthist ah
            JOIN alerttype at USING (alerttypeid)
        WHERE
            alerttype ILIKE '%upgrade'
        ORDER BY
            start_time DESC
        LIMIT 1);

An example of a plugin that copies over the varmap is this plugin (for maintenance events, see line 42 for the defining moment):

nav/python/nav/eventengine/plugins/maintenancestate.py

Lines 37 to 46 in bec0462

	def _post_alert(self, event):
	alert = AlertGenerator(event)
	alert.alert_type = (
	'onMaintenance' if event.state == event.STATE_START else 'offMaintenance'
	)
	alert.history_vars = dict(alert)
	if alert.is_event_duplicate():
	self._logger.info('Ignoring duplicate event')
	else:
	alert.post()

You can confirm this for maintenance events by re-using the above SQL:

SELECT
    *
FROM
    alerthistvar
WHERE
    alerthistid IN (
        SELECT
            ah.alerthistid
        FROM
            alerthist ah
            JOIN alerttype at USING (alerttypeid)
        WHERE
            alerttype ILIKE '%maintenance'
        ORDER BY
            start_time DESC
        LIMIT 1);

So: I think we need a new eventengine plugin before this PR makes sense. Are you up to the task? :)

[johannaengland]

I have checked this manually in combination with #2515 and it works as expected.

Now only sub-device id/name and sub-device serial number are missing, any idea on how to get that?

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[lunkwill42]

Yes, this is still going in the right direction :)

To add more information about entities, you need to join the netboxentity table, which describes the internal entities in a device.

One silly detail may be that netboxentity uses raw entPhysicalClass values from ENTITY-MIB, and there is no translation table in the database, so you may need to translate the values back into strings in order for the sub-device listing to make sense (see the cisco_serials report for a short example that only translates two of the possible values)