Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collect Juniper dom threshold values #2513

Draft
wants to merge 24 commits into
base: master
Choose a base branch
from

Conversation

stveit
Copy link
Contributor

@stveit stveit commented Nov 24, 2022

Fixes #2340

Collects threshold values as sensors. Adds new fields to the Sensor class used to define the properties of the threshold.
Adds fields for stating if the threshold should trigger for the value being higher or lower than the given threshold, and
if the severity of the alarm that should be made if the threshold is breached.
Adds properties for getting the sensor another sensor is a threshold for, and the other way around for getting all the sensors that are thresholds for a given sensor.

draft because commit log needs fixing

@stveit stveit changed the title Juniper dom threshold values Collect Juniper dom threshold values Nov 24, 2022
makes it so the graphite metric paths have generic names,
instead of names specific to the juniper columns.
Any sensor threshold will have the same naming,
no matter what type of sensor it is. all you need
to know is the netbox, the sensor and the type of
threshold you want (high alarm, low alarm etc)
@stveit stveit force-pushed the feature.juniper-dom-threshold-values branch from 46d2a77 to ebc385d Compare November 24, 2022 08:16
@sonarcloud
Copy link

sonarcloud bot commented Nov 24, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@codecov
Copy link

codecov bot commented Nov 24, 2022

Codecov Report

Merging #2513 (a9a6080) into master (fe217e2) will increase coverage by 0.49%.
The diff coverage is 62.79%.

@@            Coverage Diff             @@
##           master    #2513      +/-   ##
==========================================
+ Coverage   53.31%   53.80%   +0.49%     
==========================================
  Files         554      558       +4     
  Lines       40315    40624     +309     
==========================================
+ Hits        21494    21859     +365     
+ Misses      18821    18765      -56     
Impacted Files Coverage Δ
python/nav/ipdevpoll/plugins/sensors.py 54.68% <0.00%> (-1.32%) ⬇️
python/nav/models/manage.py 71.62% <59.09%> (+0.45%) ⬆️
python/nav/mibs/juniper_dom_mib.py 84.61% <77.77%> (+32.61%) ⬆️
python/nav/web/sortedstats/forms.py 74.07% <0.00%> (-0.93%) ⬇️
python/nav/ipdevpoll/shadows/__init__.py 41.87% <0.00%> (-0.78%) ⬇️
python/nav/web/sortedstats/statmodules.py 41.53% <0.00%> (-0.65%) ⬇️
python/nav/web/api/v1/auth.py 90.66% <0.00%> (-0.25%) ⬇️
python/nav/ipdevpoll/plugins/statsystem.py 29.13% <0.00%> (-0.20%) ⬇️
python/nav/arnold.py 57.61% <0.00%> (-0.11%) ⬇️
python/nav/models/api.py 96.42% <0.00%> (ø)
... and 34 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@github-actions
Copy link

github-actions bot commented Nov 24, 2022

Test results

     12 files       12 suites   11m 23s ⏱️
3 220 tests 3 124 ✔️   96 💤 0
9 135 runs  8 847 ✔️ 288 💤 0

Results for commit a9a6080.

♻️ This comment has been updated with latest results.

@stveit stveit self-assigned this Nov 24, 2022
Copy link
Member

@lunkwill42 lunkwill42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like much of what you've done here, but I do have some reservations about the overall solution.

First of all, the statsensors plugin is built to be very generic in what it does: It collects the sensor OIDs that have been registered in the database, period. Adding MIB mapping for threshold collection to the very same plugin introduces a new kind of complexity to the plugin that I don't care too much for.

In principle, a threshold value is just "another sensor" when it comes to collection. NAV also doesn't care that the value is a "threshold" - that interpretation is up to the user who is building their Graphite expressions for threshold rules.

The special case for threshold sensors is this: They have a relationship to another sensor (and, of course, their value is likely to be a constant - but this is not important for the implementation).

Sensor objects in NAV can have an optional relationship to an interface. I'm thinking I would like to see a solution in where a Sensor can have an optional relationship to another Sensor. Maybe something like Sensor.threshold_for which would be a relationship to the sensor that this sensor is a threshold value for.

What do you think?

Comment on lines 85 to 86
for sensor in sensors:
self._logger.info(f"Internal names: {sensor.get('internal_name', None)}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. This looks more like debug-level stuff, not info-level (not very interesting to a "common" user)
  2. Best practice for Python loggers is to use regular string formatting and expansion arguments. This ensure that the formatted string is only built if the log statement matches the currently configured log level (if debug arguments are expensive to format to strings, it doesn't happen unless DEBUG level logging is actually enabled)
  3. No need to specify a fallback argument of None when using .get() on a dict - None is already the default fallback value.
Suggested change
for sensor in sensors:
self._logger.info(f"Internal names: {sensor.get('internal_name', None)}")
for sensor in sensors:
self._logger.debug("Internal names: %s", sensor.get('internal_name'))

@defer.inlineCallbacks
def _handle_thresholds(self, sensors, netboxes):
metrics = yield self._collect_thresholds(netboxes, sensors)
self._logger.info(f"Metrics: {metrics}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about debug logging and argument expansion applies here.

Comment on lines +2378 to +2379
threshold_alert_type = models.IntegerField(
db_column='threshold_alert_type', choices=ALERT_TYPE_CHOICES, null=True
Copy link
Contributor Author

@stveit stveit Feb 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reuses pre-existing alert types. Figured this could give some consistency, as the already existing "Warning" and "Alert" match quite well with the "Warning" and "Alarm" naming scheme from the juniper thresholds.

@stveit stveit marked this pull request as draft February 15, 2023 09:37
threshold_alert_type = models.IntegerField(
db_column='threshold_alert_type', choices=ALERT_TYPE_CHOICES, null=True
)
threshold_for_oid = VarcharField(db_column='threshold_for_oid', null=True)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would make sense to be a ForeignKey to the Sensor object this is a threshold for, but it didnt seem very easy to do. The values for the sensor object that is to be created are set after discovery, but before any objects are actually made, so refering to a sensor object that doesnt exist yet seems difficult. But pairing this threshold_for_oid field with a property function threshold_for that gives the actual sensor seemed like the next best thing.

@sonarcloud
Copy link

sonarcloud bot commented Feb 21, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Collect threshold values from JUNIPER-DOM-MIB
2 participants