Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Next 2.0 #575

Merged
merged 70 commits into from
Oct 1, 2023
Merged
Changes from 3 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
8eb97ee
Update version statements
jmcgill298 Jul 28, 2023
65ce1f2
Replace broken imports
jmcgill298 Jul 31, 2023
ff5e5a2
Swap Region/Site to Location
jmcgill298 Aug 9, 2023
914054c
Fix/skip test cases
jmcgill298 Aug 14, 2023
f93a72f
Merge pull request #547 from nautobot/jmcgill/NP-62/modify-import-sta…
jmcgill298 Sep 5, 2023
b808c3a
Change filters for 2.x
jmcgill298 Sep 6, 2023
f5026cf
Merge pull request #565 from nautobot/jmcgill/2.x-filters
jmcgill298 Sep 6, 2023
d74a53d
Remove scope property
jmcgill298 Sep 6, 2023
bc4d9f7
Merge pull request #568 from nautobot/jmcgill/2.x-remove-scope-property
jmcgill298 Sep 6, 2023
0d12d61
Remove code around csv import/export per 2.x changes
jmcgill298 Sep 2, 2023
ab7cb45
Merge pull request #569 from nautobot/jmcgill/2.x-remove-csv-import-e…
jmcgill298 Sep 6, 2023
571dd95
Change some platform slug uses to network_driver
jmcgill298 Sep 6, 2023
93b95e0
Remove remaining references to slugs in core models
jmcgill298 Sep 7, 2023
537c16f
Merge pull request #570 from nautobot/jmcgill/2.x-remove-slugs
jmcgill298 Sep 7, 2023
b912a14
Switch tag filter to tags
jmcgill298 Sep 7, 2023
82fe01e
Merge pull request #571 from nautobot/jmcgill/2.x-tag-to-tags
jmcgill298 Sep 7, 2023
7158511
Remove custom get_absolute_url method
jmcgill298 Sep 7, 2023
2e7e29a
Merge pull request #572 from nautobot/jmcgill/2.x-use-default-get-abs…
jmcgill298 Sep 7, 2023
ba064e4
Merge branch 'next' into next-2.0
itdependsnetworks Sep 8, 2023
8773faf
Remove management command, move to new dispatcher method, fix job log…
itdependsnetworks Sep 10, 2023
718b18e
Cookie initialy baked by NetworkToCode Cookie Drift Manager Tool (#583)
snaselj Sep 13, 2023
2096a5c
Jmcgill/2.x jobs rebase (#595)
jmcgill298 Sep 16, 2023
e5d400c
todos updates (#598)
itdependsnetworks Sep 16, 2023
59f04ee
Merge branch 'next-2.0' into next
itdependsnetworks Sep 16, 2023
649bead
Merge pull request #599 from itdependsnetworks/next-2.0-potential
itdependsnetworks Sep 16, 2023
9ab2d8f
stopping point
itdependsnetworks Sep 16, 2023
580aa44
More updates for 2.0
itdependsnetworks Sep 17, 2023
95d2e83
Move to new dispatcher, move compliant to device tab, update location…
itdependsnetworks Sep 18, 2023
01f8cf0
bump versions
itdependsnetworks Sep 18, 2023
75dbdd4
remove platform mapping.
itdependsnetworks Sep 18, 2023
c244736
Remove slugs added by Config Remediation work (#602)
jmcgill298 Sep 19, 2023
1c2d7ad
Logging updates, active tab fix, add default_deploy_status,
itdependsnetworks Sep 21, 2023
b1054e1
Merge branch 'next-2.0' into next-merge-conflicts
itdependsnetworks Sep 23, 2023
49eaf77
Merge pull request #614 from nautobot/next-merge-conflicts
itdependsnetworks Sep 23, 2023
3534ead
fix links
itdependsnetworks Sep 21, 2023
5830850
Migrate config compliance and golden config views to viewsets, upgrad…
itdependsnetworks Sep 25, 2023
017d5e1
General cleanup from linters (#615)
jmcgill298 Sep 27, 2023
c2fbe75
Revert list view actions to views, fix logging, dispatcher, call jobs…
itdependsnetworks Sep 27, 2023
88c86b2
Re-lock, and use hyperlinked_object
itdependsnetworks Sep 28, 2023
eb554d1
fix GoldenConfigSettingForm
HanlinMiao Sep 28, 2023
a9412cd
downgrade django-pivot to 1.8.1
HanlinMiao Sep 28, 2023
843f920
Fix failing notes_url test (#625)
gsnider2195 Sep 28, 2023
35f87ec
bump version, remove slugify, remove commit from config deploy, simpl…
itdependsnetworks Sep 28, 2023
cffb979
Fix some more tests for 2.0 (#631)
gsnider2195 Sep 29, 2023
2a03d2f
Update git, logger bug, update processor
itdependsnetworks Sep 29, 2023
9aef080
logger updates, add to request object, linter
itdependsnetworks Sep 29, 2023
276b558
audit linkable retrieve links, link configs vs show them
itdependsnetworks Sep 29, 2023
74e0adf
fix status link
itdependsnetworks Sep 29, 2023
6700ab2
Fix execute menu, extra context in GC overview, expanded to support RC4
bryanculver Sep 29, 2023
2820741
update gitrepo flow, switch to core get_app_settings_or_config, updat…
itdependsnetworks Sep 29, 2023
d49a25e
Invert Completed logic
itdependsnetworks Sep 29, 2023
cfb8c3d
fix git tests
gsnider2195 Sep 29, 2023
78d640d
Rename network_driver_mapper to network_driver_mappings (#639)
mzbroch Sep 29, 2023
78b5bb4
Add logic to warn when GoldenConfig is out of sync (#629)
jmcgill298 Sep 29, 2023
73fa533
update & clean up processor logic, fix reference to .url
itdependsnetworks Sep 29, 2023
96611a2
fix datasources yaml key to network_driver but backwards to _slug
jeffkala Sep 29, 2023
f939026
fix datasources
jeffkala Sep 29, 2023
25bcb64
fix js issue where was too greedy in selection, add copy button
itdependsnetworks Sep 29, 2023
d96f64a
few additional fixes adn cleanups
jeffkala Sep 29, 2023
88eead8
Merge branch 'next-2.0' of github.com:nautobot/nautobot-plugin-golden…
jeffkala Sep 29, 2023
9c59044
fix black
jeffkala Sep 29, 2023
481c380
fix filtering on configplans
jeffkala Sep 29, 2023
81db93d
Add back constance code and clean up json view code (#642)
itdependsnetworks Sep 30, 2023
8a3bd1b
version update and remove json/yaml toggle
itdependsnetworks Sep 30, 2023
e4f2944
linting updates
itdependsnetworks Sep 30, 2023
eb02065
fix api that populates filterform for plan and deploy results
jeffkala Sep 30, 2023
a7d5647
Start on migration/release docs, make booleans more consistent, add e…
itdependsnetworks Sep 30, 2023
d40b64a
Migration docs updates, 2.0 release notes, remove dispatcher references
itdependsnetworks Oct 1, 2023
70aa4b3
add custom dispatcher docs, update secrets docs, add constance docs, …
itdependsnetworks Oct 1, 2023
f5f4a91
update dependencies, fix or skip tests, minor doc updates
itdependsnetworks Oct 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions development/nautobot_config.py
Original file line number Diff line number Diff line change
@@ -181,11 +181,11 @@
"trim_blocks": is_truthy(os.getenv("NAUTOBOT_JINJA_ENV_TRIM_BLOCKS", True)),
"lstrip_blocks": is_truthy(os.getenv("NAUTOBOT_JINJA_ENV_LSTRIP_BLOCKS", False)),
},
# The platform_slug_map maps an arbitrary platform slug to its corresponding parser.
# Use this if the platform slug names in your Nautobot instance don't correspond exactly
# The platform_network_driver_map maps an arbitrary platform network_driver to its corresponding parser.
# Use this if the platform network_driver names in your Nautobot instance don't correspond exactly
# to the Nornir driver names ("arista_eos", "cisco_ios", etc.).
# Each key should == the slug of the Nautobot platform object.
# "platform_slug_map": {
# Each key should == the network_driver of the Nautobot platform object.
# "platform_network_driver_map": {
# "eos": "arista_eos",
# "ios": "cisco_ios",
# "iosxe": "cisco_ios",
32 changes: 16 additions & 16 deletions docs/admin/admin_install.md
Original file line number Diff line number Diff line change
@@ -58,7 +58,7 @@ PLUGINS_CONFIG = {
"enable_postprocessing": False,
"postprocessing_callables": [],
"postprocessing_subscribed": [],
"platform_slug_map": None,
"platform_network_driver_map": None,
"jinja_env": {
"undefined": StrictUndefined, # jinja2.StrictUndefined
"trim_blocks": True,
@@ -92,21 +92,21 @@ The plugin behavior can be controlled with the following list of settings.
!!! note
The `enable_backup`, `enable_compliance`, `enable_intended`, `enable_sotagg` and `enable_postprocessing` will toggle inclusion of the entire component.

| Key | Example | Default | Description |
| ------------------------- | ----------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| enable_backup | True | True | A boolean to represent whether or not to run backup configurations within the plugin. |
| enable_compliance | True | True | A boolean to represent whether or not to run the compliance process within the plugin. |
| enable_intended | True | True | A boolean to represent whether or not to generate intended configurations within the plugin. |
| enable_sotagg | True | True | A boolean to represent whether or not to provide a GraphQL query per device to allow the intended configuration to provide data variables to the plugin. |
| enable_postprocessing | True | False | A boolean to represent whether or not to generate intended configurations to push, with extra processing such as secrets rendering. |
| postprocessing_callables | ['mypackage.myfunction'] | [] | A list of function paths, in dotted format, that are appended to the available methods for post-processing the intended configuration, for instance, the `render_secrets`. |
| postprocessing_subscribed | ['mypackage.myfunction'] | [] | A list of function paths, that should exist as postprocessing_callables, that defines the order of application of during the post-processing process. |
| platform_slug_map | {"cisco_wlc": "cisco_aireos"} | None | A dictionary in which the key is the platform slug and the value is what netutils uses in any "network_os" parameter within `netutils.config.compliance.parser_map`. |
| sot_agg_transposer | "mypkg.transposer" | None | A string representation of a function that can post-process the graphQL data. |
| per_feature_bar_width | 0.15 | 0.15 | The width of the table bar within the overview report |
| per_feature_width | 13 | 13 | The width in inches that the overview table can be. |
| per_feature_height | 4 | 4 | The height in inches that the overview table can be. |
| jinja_env | {"lstrip_blocks": False} | See Note Below | A dictionary of Jinja2 Environment options compatible with Jinja2.SandboxEnvironment() |
| Key | Example | Default | Description |
| ----------------------------------- | ----------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| enable_backup | True | True | A boolean to represent whether or not to run backup configurations within the plugin. |
| enable_compliance | True | True | A boolean to represent whether or not to run the compliance process within the plugin. |
| enable_intended | True | True | A boolean to represent whether or not to generate intended configurations within the plugin. |
| enable_sotagg | True | True | A boolean to represent whether or not to provide a GraphQL query per device to allow the intended configuration to provide data variables to the plugin. |
| enable_postprocessing | True | False | A boolean to represent whether or not to generate intended configurations to push, with extra processing such as secrets rendering. |
| postprocessing_callables | ['mypackage.myfunction'] | [] | A list of function paths, in dotted format, that are appended to the available methods for post-processing the intended configuration, for instance, the `render_secrets`. |
| postprocessing_subscribed | ['mypackage.myfunction'] | [] | A list of function paths, that should exist as postprocessing_callables, that defines the order of application of during the post-processing process. |
| platform_network_driver_map | {"cisco_wlc": "cisco_aireos"} | None | A dictionary in which the key is the platform slug and the value is what netutils uses in any "network_os" parameter within `netutils.config.compliance.parser_map`. |
| sot_agg_transposer | "mypkg.transposer" | None | A string representation of a function that can post-process the graphQL data. |
| per_feature_bar_width | 0.15 | 0.15 | The width of the table bar within the overview report |
| per_feature_width | 13 | 13 | The width in inches that the overview table can be. |
| per_feature_height | 4 | 4 | The height in inches that the overview table can be. |
| jinja_env | {"lstrip_blocks": False} | See Note Below | A dictionary of Jinja2 Environment options compatible with Jinja2.SandboxEnvironment() |

!!! note
Over time the compliance report will become more dynamic, but for now allow users to configure the `per_*` configs in a way that fits best for them.
10 changes: 5 additions & 5 deletions docs/user/app_faq.md
Original file line number Diff line number Diff line change
@@ -40,7 +40,7 @@ Understanding that there will never be consensus on what should go into a featur

## _What are the supported platforms for Compliance jobs? How do I configure a device with a specific OS?_

The current supported platform and the associated *default* platform slug names are the following for:
The current supported platform and the associated *default* platform network_driver names are the following for:

* arista_eos
* aruba_aoscx
@@ -60,12 +60,12 @@ The current supported platform and the associated *default* platform slug names
* nokia_sros
* paloalto_panos

The expected "network_os" parameter must be as defined by netutils and golden config uses the platform slug to map from the device to the appropriate "network_os" that netutils expects. However, there an ability to map the actual platform slug for compliance and parsing tasks via the plugin settings in your "nautobot_config.py", and documented on the primary Readme.
The expected "network_os" parameter must be as defined by netutils and golden config uses the platform network_driver to map from the device to the appropriate "network_os" that netutils expects. However, there an ability to map the actual platform network_driver for compliance and parsing tasks via the plugin settings in your "nautobot_config.py", and documented in [App Configuration](../admin/admin_install.md#app-configuration).

To provide a concrete example of this, note the following example that demonstrates how you can transpose any platform slug name to the expected one, as well as map multiple keys to a single netutils expected key. The `platform_slug_map` is only used for configuration compliance job. The json key is the Nautobot platform slug, and the json value is the "network_os" parameter defined in `netutils.config.compliance.parser_map`.
To provide a concrete example of this, note the following example that demonstrates how you can transpose any platform network_driver name to the expected one, as well as map multiple keys to a single netutils expected key. The `platform_network_driver_map` is only used for configuration compliance job. The json key is the Nautobot platform network_driver, and the json value is the "network_os" parameter defined in `netutils.config.compliance.parser_map`.
```json
{
"platform_slug_map": {
"platform_network_driver_map": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this still? With core having network drivers now that link to netmiko , we no longer care about slug right?

"cisco_aireos": "cisco_wlc",
"ios": "cisco_ios",
"iosxe": "cisco_ios"
@@ -75,7 +75,7 @@ To provide a concrete example of this, note the following example that demonstra

## _What are the supported platforms for Backup and Intended Configuration jobs? How do I configure a device with a specific OS?_

The current supported platform and the associated *default* platform slug names are the following for:
The current supported platform and the associated *default* platform network_driver names are the following for:

* arista_eos
* cisco_asa
4 changes: 2 additions & 2 deletions docs/user/app_feature_compliance.md
Original file line number Diff line number Diff line change
@@ -48,7 +48,7 @@ Each configuration can be added and edits from this table. When editing/adding t

![Configuration Rule Edit](../images/ss_compliance-rule.png)

The platform must refer to a platform with a valid slug supported by the configuration compliance engine. While there is no enforcement of this data from
The platform must refer to a platform with a valid network_driver supported by the configuration compliance engine. While there is no enforcement of this data from
a database perspective, the job will never run successfully, rendering the additional configuration ineffective.

The Feature is a unique identifier, that should prefer shorter names, as this effects the width of the compliance overview and thus it's readability as a
@@ -90,7 +90,7 @@ Please note the following about the compliance details page.
## Supported Platforms

Platforms support technically come from the options provided by [nornir-nautobot](https://github.com/nautobot/nornir-nautobot) for Nornir dispatcher tasks and
[netutils](https://github.com/networktocode/netutils) for configuration compliance and parsing. However, for reference, the valid slug's of the platforms are
[netutils](https://github.com/networktocode/netutils) for configuration compliance and parsing. However, for reference, the valid network_driver's of the platforms are
provided in the [FAQ](./app_faq).

## Overview Report
4 changes: 2 additions & 2 deletions docs/user/app_feature_compliancecustom.md
Original file line number Diff line number Diff line change
@@ -35,7 +35,7 @@ The interface of contract provided to your custom function is based on the follo
- The `obj` parameter, is the `self` instance object of a `ConfigCompliance` model, review the documentation for the all attributes of a `ConfigCompliance` instance, but the common ones are documented below.
- `obj.actual` - The **actual** configuration parsed out by the `match_config` logic, or what was sent via the API.
- `obj.intended` - The **intended** configuration parsed out by the `match_config` logic, or what was sent via the API.
- `obj.device.platform.slug` - The platform slug name.
- `obj.device.platform.network_driver` - The platform network_driver name.
- `obj.rule.config_ordered` - describes whether or not the rule was configured to be ordered, such as an ACL, or not such as SNMP servers
- `obj.rule` - The name of the rule.
- `obj.rule.match_config` - The match_config text the rule was configured with.
@@ -104,7 +104,7 @@ import re
BGP_PATTERN = re.compile("\s*neighbor (?P<ip>\d+\.\d+\.\d+\.\d+) .*")
BGP_SECRET = re.compile("\s*neighbor (?P<ip>\d+\.\d+\.\d+\.\d+) password (\S+).*")
def custom_compliance_func(obj):
if obj.rule == 'bgp' and obj.device.platform.slug == 'ios':
if obj.rule == 'bgp' and obj.device.platform.network_driver == 'cisco_ios':
actual_config = obj.actual
neighbors = []
secrets = []
16 changes: 8 additions & 8 deletions docs/user/app_feature_config_postprocessing.md
Original file line number Diff line number Diff line change
@@ -38,18 +38,18 @@ There are two different ways to customize the default behavior of `get_config_po

The `render_secrets` function performs an extra Jinja rendering on top of an intended configuration, exposing new custom Jinja filters:

- `get_secret_by_secret_group_slug`: as the name suggests, it returns the secret_group value, for a secret type, from its `slug`.
- `get_secret_by_secret_group_name`: as the name suggests, it returns the secret_group value, for a secret type, from its `name`.

!!! note
Other default Django or Netutils filters are not available in this Jinja environment. Only `encrypt_<vendor>_type5` and `encrypt_<vendor>_type7` can be used together with the `get_secret` filters.

Because this rendering is separated from the standard generation of the intended configuration, you must use the `{% raw %}` Jinja syntax to avoid being processed by the initial generation stage.

1. For example, an original template like this, `{% raw %}ppp pap sent-username {{ secrets_group["slug"] | get_secret_by_secret_group_slug("username")}}{% endraw %}`
2. Produces an intended configuration as `ppp pap sent-username {{ secrets_group["slug"] | get_secret_by_secret_group_slug("username") }}`
1. For example, an original template like this, `{% raw %}ppp pap sent-username {{ secrets_group["name"] | get_secret_by_secret_group_name("username")}}{% endraw %}`
2. Produces an intended configuration as `ppp pap sent-username {{ secrets_group["name"] | get_secret_by_secret_group_name("username") }}`
3. After the `render_secrets`, it becomes `ppp pap sent-username my_username`.

Notice that the `get_secret` filters take arguments. In the example, the `Secret_group` slug is passed, together with the type of the `Secret`. Check every signature for extra customization.
Notice that the `get_secret` filters take arguments. In the example, the `secret_group` name is passed, together with the type of the `Secret`. Check every signature for extra customization.

!!! note
Remember that to render these secrets, the user requesting it via UI or API, MUST have read permissions to Secrets Groups, Golden Config, and the specific Device object.
@@ -64,7 +64,7 @@ This shows how Render the Secrets feature for a `Device`, for the default `Secre
query ($device_id: ID!) {
device(id: $device_id) {
secrets_group {
slug
name
}
location {
rel_my_secret_relationship_for_location {
@@ -80,13 +80,13 @@ query ($device_id: ID!) {
Using the default `secrets_group` FK in `Device`:

```jinja2
{% raw %}{{ secrets_group["slug"] | get_secret_by_secret_group_slug("password") | default('no password') }}{% endraw %}
{% raw %}{{ secrets_group["name"] | get_secret_by_secret_group_name("password") | default('no password') }}{% endraw %}
```

Using the custom relationship at the `Location` level:

```jinja2
{% raw %}{{ location["rel_my_secret_relationship_for_location"][0]["slug"] | get_secret_by_secret_group_slug("password") | default('no password') }}{% endraw %}
{% raw %}{{ location["rel_my_secret_relationship_for_location"][0]["name"] | get_secret_by_secret_group_name("password") | default('no password') }}{% endraw %}
```

This will end up rendering the secret, of type "password", for the corresponding `SecretGroup`.
@@ -96,5 +96,5 @@ This will end up rendering the secret, of type "password", for the corresponding
Obviously, the rendering process can find multiple challenges, that are managed, and properly explained to take corrective actions:

```
Found an error rendering the configuration to push: Jinja encountered and UndefinedError: 'None' has no attribute 'slug', check the template for missing variable definitions.
Found an error rendering the configuration to push: Jinja encountered and UndefinedError: 'None' has no attribute 'name', check the template for missing variable definitions.
```
9 changes: 4 additions & 5 deletions docs/user/app_feature_sotagg.md
Original file line number Diff line number Diff line change
@@ -28,9 +28,9 @@ operator to point to a function within the python path by a string. The function
```python
def transposer(data):
"""Some."""
if data["platform"]["slug"] == "cisco_ios":
if data["platform"]["network_driver"] == "cisco_ios":
data["platform"].update({"support-number": "1-800-ciscohelp"})
if data["platform"]["slug"] == "arista_eos":
if data["platform"]["network_driver"] == "arista_eos":
data["platform"].update({"support-number": "1-800-aristahelp"})
return data
```
@@ -92,17 +92,16 @@ query ($device_id: ID!) {
}
tags {
name
slug
}
device_role {
role {
name
}
platform {
name
slug
manufacturer {
name
}
network_driver
napalm_driver
}
location {
Loading