feat: Add option to provide collectors for hardware dependent tests #168
+231
−28
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dashmage
force-pushed
the
bseng-1381-add-collector-option
branch
4 times, most recently
from
3fd8293 to
0689dfd
Compare
dashmage
commented
Feb 21, 2024
dashmage
force-pushed
the
bseng-1381-add-collector-option
branch
2 times, most recently
from
3ff2a28 to
9e6391f
Compare
sudeephb
reviewed
Feb 22, 2024
sudeephb
reviewed
Feb 22, 2024
dashmage
force-pushed
the
bseng-1381-add-collector-option
branch
2 times, most recently
from
2f17a32 to
be3eba9
Compare
dashmage
changed the title
rgildein
suggested changes
Feb 26, 2024
dashmage
force-pushed
the
bseng-1381-add-collector-option
branch
3 times, most recently
from
06fca3c to
c6711ca
Compare
Also adds "realhw" marker which when applied to test functions, will skip them in case no collectors are provided via the CLI option.
These tests were added: * collector was enabled in hardware-exporter-config.yaml file * metrics are available at the expected endpoint on the unit * collector specific metrics are also available at the endpoint based on the collectors options provided. Also added helper functions and a dataclass for obtaining metrics data and parsing it.
* Move helper functions from SyncHelper to utils.py * Obtain default exporter port constant from config.py * Refactor parse_metrics to simplify logic * Change test_config_collector_enabled so that it passes only when all enabled collectors for the hardware are provided
dashmage
force-pushed
the
bseng-1381-add-collector-option
branch
from
c6711ca to
0557fab
Compare
rgildein
approved these changes
Feb 28, 2024
jneo8
reviewed
Feb 29, 2024
This was referenced Mar 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is part of the effort to automate many of the manual tests for hardware observer. These tests depend on the availability of actual hardware resources. The tests are not meant to be run as part of CI but provide an easier way to validate functionality with real hardware whenever required, for example, before a new release.
The tests include:
Collectors that need to be tested (as per availability on the underlying hardware) can be provided via the
--collectorsflag as space separated arguments. Eg:--collectors ipmi_sel redfish mega_raid. These collectors need to belong to a pre-configured list present inconftest.py.The
realhwmarker has also been added to mark the tests which are hardware dependent. If the--collectorsflag is not provided, then all these tests will be skipped so that only the regular, hardware independent tests are executed.A
Metricdataclass has been created to hold the metric data to facilitate testing. The accompanying parsing functions help to convert the raw string of metrics into this structure. Currently,test_collector_specific_metrics_availableutilizes this to check whether the metrics for the provided collectors are present in the metrics output. In future, this can be used to test for specific value and labels for the metrics.A new
utils.pymodule has been added to collect all the helper functions (including the ones from theSyncHelperclass which have been moved here).Currently, running these hardware dependent tests require adding the machine to the model beforehand and performing the deployments manually for
hardware-observer,grafana-agentandubuntuapplications. The modifications to thetest_build_and_deployfunction for automating this via the bundle and adding the required juju resources will be handled in another PR.