Merge pull request #11180 from netbox-community/develop

Release v3.4.0

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v3.3.10
+      placeholder: v3.4.0
     validations:
       required: true
   - type: dropdown
@@ -25,6 +25,7 @@ body:
         - "3.8"
         - "3.9"
         - "3.10"
+        - "3.11"
     validations:
       required: true
   - type: textarea

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v3.3.10
+      placeholder: v3.4.0
     validations:
       required: true
   - type: dropdown

.github/workflows/ci.yml

@@ -9,7 +9,7 @@ jobs:
       NETBOX_CONFIGURATION: netbox.configuration_testing
     strategy:
       matrix:
-        python-version: ['3.8', '3.9', '3.10']
+        python-version: ['3.8', '3.9', '3.10', '3.11']
         node-version: ['14.x']
     services:
       redis:

base_requirements.txt

@@ -4,7 +4,7 @@ bleach
 
 # The Python web framework on which NetBox is built
 # https://github.com/django/django
-Django<4.1
+Django<4.2
 
 # Django middleware which permits cross-domain API requests
 # https://github.com/OttoYiu/django-cors-headers
@@ -68,7 +68,7 @@ drf-yasg[validation]
 
 # Django wrapper for Graphene (GraphQL support)
 # https://github.com/graphql-python/graphene-django
-graphene_django<3.0
+graphene_django
 
 # WSGI HTTP server
 # https://gunicorn.org/

docs/configuration/data-validation.md

@@ -58,16 +58,19 @@ The following model fields support configurable choices:
 * `circuits.Circuit.status`
 * `dcim.Device.status`
 * `dcim.Location.status`
+* `dcim.Module.status`
 * `dcim.PowerFeed.status`
 * `dcim.Rack.status`
 * `dcim.Site.status`
+* `dcim.VirtualDeviceContext.status`
 * `extras.JournalEntry.kind`
 * `ipam.IPAddress.status`
 * `ipam.IPRange.status`
 * `ipam.Prefix.status`
 * `ipam.VLAN.status`
 * `virtualization.Cluster.status`
 * `virtualization.VirtualMachine.status`
+* `wireless.WirelessLAN.status`
 
 The following colors are supported:
 

docs/configuration/miscellaneous.md

@@ -141,6 +141,22 @@ When determining the primary IP address for a device, IPv6 is preferred over IPv
 
 ---
 
+## QUEUE_MAPPINGS
+
+Allows changing which queues are used internally for background tasks.
+
+```python
+QUEUE_MAPPINGS = {
+    'webhook': 'low',
+    'report': 'high',
+    'script': 'high',
+}
+```
+
+If no queue is defined the queue named `default` will be used.
+
+---
+
 ## RELEASE_CHECK_URL
 
 Default: None (disabled)

docs/configuration/required-parameters.md

@@ -25,7 +25,7 @@ ALLOWED_HOSTS = ['*']
 
 ## DATABASE
 
-NetBox requires access to a PostgreSQL 10 or later database service to store data. This service can run locally on the NetBox server or on a remote system. The following parameters must be defined within the `DATABASE` dictionary:
+NetBox requires access to a PostgreSQL 11 or later database service to store data. This service can run locally on the NetBox server or on a remote system. The following parameters must be defined within the `DATABASE` dictionary:
 
 * `NAME` - Database name
 * `USER` - PostgreSQL username

docs/configuration/security.md

@@ -1,5 +1,13 @@
 # Security & Authentication Parameters
 
+## ALLOW_TOKEN_RETRIEVAL
+
+Default: True
+
+If disabled, the values of API tokens will not be displayed after each token's initial creation. A user **must** record the value of a token immediately upon its creation, or it will be lost. Note that this affects _all_ users, regardless of assigned permissions.
+
+---
+
 ## ALLOWED_URL_SCHEMES
 
 !!! tip "Dynamic Configuration Parameter"

docs/configuration/system.md

@@ -157,6 +157,14 @@ The file path to the location where [custom scripts](../customization/custom-scr
 
 ---
 
+## SEARCH_BACKEND
+
+Default: `'netbox.search.backends.CachedValueSearchBackend'`
+
+The dotted path to the desired search backend class. `CachedValueSearchBackend` is currently the only search backend provided in NetBox, however this setting can be used to enable a custom backend. 
+
+---
+
 ## STORAGE_BACKEND
 
 Default: None (local storage)

docs/customization/custom-fields.md

@@ -13,6 +13,7 @@ Custom fields may be created by navigating to Customization > Custom Fields. Net
 * Text: Free-form text (intended for single-line use)
 * Long text: Free-form of any length; supports Markdown rendering
 * Integer: A whole number (positive or negative)
+* Decimal: A fixed-precision decimal number (4 decimal places)
 * Boolean: True or false
 * Date: A date in ISO 8601 format (YYYY-MM-DD)
 * URL: This will be presented as a link in the web UI

docs/customization/custom-scripts.md

@@ -267,7 +267,7 @@ An IPv4 or IPv6 network with a mask. Returns a `netaddr.IPNetwork` object. Two a
 
 ### Via the Web UI
 
-Custom scripts can be run via the web UI by navigating to the script, completing any required form data, and clicking the "run script" button.
+Custom scripts can be run via the web UI by navigating to the script, completing any required form data, and clicking the "run script" button. It is possible to schedule a script to be executed at specified time in the future. A scheduled script can be canceled by deleting the associated job result object.
 
 ### Via the API
 
@@ -282,6 +282,8 @@ http://netbox/api/extras/scripts/example.MyReport/ \
 --data '{"data": {"foo": "somevalue", "bar": 123}, "commit": true}'
 ```
 
+Optionally `schedule_at` can be passed in the form data with a datetime string to schedule a script at the specified date and time.
+
 ### Via the CLI
 
 Scripts can be run on the CLI by invoking the management command:

docs/customization/reports.md

@@ -136,7 +136,7 @@ Once you have created a report, it will appear in the reports list. Initially, r
 
 ### Via the Web UI
 
-Reports can be run via the web UI by navigating to the report and clicking the "run report" button at top right. Once a report has been run, its associated results will be included in the report view.
+Reports can be run via the web UI by navigating to the report and clicking the "run report" button at top right. Once a report has been run, its associated results will be included in the report view. It is possible to schedule a report to be executed at specified time in the future. A scheduled report can be canceled by deleting the associated job result object.
 
 ### Via the API
 
@@ -152,6 +152,8 @@ Our example report above would be called as:
     POST /api/extras/reports/devices.DeviceConnectionsReport/run/
 ```
 
+Optionally `schedule_at` can be passed in the form data with a datetime string to schedule a script at the specified date and time.
+
 ### Via the CLI
 
 Reports can be run on the CLI by invoking the management command:

docs/development/adding-models.md

@@ -60,7 +60,7 @@ Create the HTML template for the object view. (The other views each typically em
 
 ## 10. Add the model to the navigation menu
 
-Add the relevant navigation menu items in `netbox/netbox/navigation_menu.py`.
+Add the relevant navigation menu items in `netbox/netbox/navigation/menu.py`.
 
 ## 11. REST API components
 

docs/development/extending-models.md

@@ -56,11 +56,15 @@ If the new field should be filterable, add it to the `FilterSet` for the model.
 
 If the new field will be included in the object list view, add a column to the model's table. For simple fields, adding the field name to `Meta.fields` will be sufficient. More complex fields may require declaring a custom column. Also add the field name to `default_columns` if the column should be present in the table by default.
 
-## 8. Update the UI templates
+## 8. Update the SearchIndex
+
+Where applicable, add the new field to the model's SearchIndex for inclusion in global search.
+
+## 9. Update the UI templates
 
 Edit the object's view template to display the new field. There may also be a custom add/edit form template that needs to be updated.
 
-## 9. Create/extend test cases
+## 10. Create/extend test cases
 
 Create or extend the relevant test cases to verify that the new field and any accompanying validation logic perform as expected. This is especially important for relational fields. NetBox incorporates various test suites, including:
 
@@ -72,6 +76,6 @@ Create or extend the relevant test cases to verify that the new field and any ac
 
 Be diligent to ensure all of the relevant test suites are adapted or extended as necessary to test any new functionality.
 
-## 10. Update the model's documentation
+## 11. Update the model's documentation
 
 Each model has a dedicated page in the documentation, at `models/<app>/<model>.md`. Update this file to include any relevant information about the new field.

docs/development/models.md

@@ -40,6 +40,7 @@ The Django [content types](https://docs.djangoproject.com/en/stable/ref/contrib/
 * [dcim.RackReservation](../models/dcim/rackreservation.md)
 * [dcim.Site](../models/dcim/site.md)
 * [dcim.VirtualChassis](../models/dcim/virtualchassis.md)
+* [dcim.VirtualDeviceContext](../models/dcim/virtualdevicecontext.md)
 * [ipam.Aggregate](../models/ipam/aggregate.md)
 * [ipam.ASN](../models/ipam/asn.md)
 * [ipam.FHRPGroup](../models/ipam/fhrpgroup.md)

docs/development/search.md

@@ -0,0 +1,37 @@
+# Search
+
+NetBox v3.4 introduced a new global search mechanism, which employs the `extras.CachedValue` model to store discrete field values from many models in a single table.
+
+## SearchIndex
+
+To enable search support for a model, declare and register a subclass of `netbox.search.SearchIndex` for it. Typically, this will be done within an app's `search.py` module.
+
+```python
+from netbox.search import SearchIndex, register_search
+
+@register_search
+class MyModelIndex(SearchIndex):
+    model = MyModel
+    fields = (
+        ('name', 100),
+        ('description', 500),
+        ('comments', 5000),
+    )
+```
+
+A SearchIndex subclass defines both its model and a list of two-tuples specifying which model fields to be indexed and the weight (precedence) associated with each. Guidance on weight assignment for fields is provided below.
+
+### Field Weight Guidance
+
+| Weight | Field Role                                       | Examples                                           |
+|--------|--------------------------------------------------|----------------------------------------------------|
+| 50     | Unique serialized attribute                      | Device.asset_tag                                   |
+| 60     | Unique serialized attribute (per related object) | Device.serial                                      |
+| 100    | Primary human identifier                         | Device.name, Circuit.cid, Cable.label              |
+| 110    | Slug                                             | Site.slug                                          |
+| 200    | Secondary identifier                             | Provider.account, DeviceType.part_number           |
+| 300    | Highly unique descriptive attribute              | CircuitTermination.xconnect_id, IPAddress.dns_name |
+| 500    | Description                                      | Site.description                                   |
+| 1000   | Custom field default                             | -                                                  |
+| 2000   | Other discrete attribute                         | CircuitTermination.port_speed                      |
+| 5000   | Comment field                                    | Site.comments                                      |

docs/features/customization.md

@@ -71,13 +71,13 @@ To learn more about this feature, check out the [export template documentation](
 
 NetBox administrators can install custom Python scripts, known as _reports_, which run within NetBox and can be executed and analyzed within the NetBox UI. Reports are a great way to evaluate NetBox objects against a set of arbitrary rules. For example, you could write a report to check that every router has a loopback interface with an IP address assigned, or that every site has a minimum set of VLANs defined.
 
-When a report runs, its logs messages pertaining to the operations being performed, and will ultimately result in either a pass or fail. Reports can be executed via the UI, REST API, or CLI (as a management command).
+When a report runs, its logs messages pertaining to the operations being performed, and will ultimately result in either a pass or fail. Reports can be executed via the UI, REST API, or CLI (as a management command). They can be run immediately or scheduled to run at a future time.
 
 To learn more about this feature, check out the [documentation for reports](../customization/reports.md).
 
 ## Custom Scripts
 
-Custom scripts are similar to reports, but more powerful. A custom script can prompt the user for input via a form (or API data), and is built to do much more than just reporting. Custom scripts are generally used to automate tasks, such as the population of new objects in NetBox, or exchanging data with external systems.
+Custom scripts are similar to reports, but more powerful. A custom script can prompt the user for input via a form (or API data), and is built to do much more than just reporting. Custom scripts are generally used to automate tasks, such as the population of new objects in NetBox, or exchanging data with external systems. As with reports, they can be run via the UI, REST API, or CLI, and be scheduled to execute at a future time.
 
 The complete Python environment is available to a custom script, including all of NetBox's internal mechanisms: There are no artificial restrictions on what a script can do. As such, custom scripting is considered an advanced feature and requires sufficient familiarity with Python and NetBox's data model.
 

docs/features/devices-cabling.md

@@ -65,6 +65,10 @@ Each device can have an operational status, functional role, and software platfo
 
 Sometimes it is necessary to model a set of physical devices as sharing a single management plane. Perhaps the most common example of such a scenario is stackable switches. These can be modeled as virtual chassis in NetBox, with one device acting as the chassis master and the rest as members. All components of member devices will appear on the master.
 
+### Virtual Device Contexts
+
+A virtual device context (VDC) is a logical partition within a device. Each VDC operates autonomously but shares a common pool of resources. Each interface can be assigned to one or more VDCs on its device.
+
 ## Module Types & Modules
 
 Much like device types and devices, module types can instantiate discrete modules, which are hardware components installed within devices. Modules often have their own child components, which become available to the parent device. For example, when modeling a chassis-based switch with multiple line cards in NetBox, the chassis would be created (from a device type) as a device, and each of its line cards would be instantiated from a module type as a module installed in one of the device's module bays.

docs/features/journaling.md

@@ -1,5 +1,5 @@
 # Journaling
 
-All primary objects in NetBox support journaling. A journal is a collection of human-generated notes and comments about an object maintained for historical context. It supplements NetBox's change log to provide additional information about why changes have been made or to convey events which occur outside NetBox. Unlike the change log, in which records typically expire after a configurable period of time, journal entries persist for the life of their associated object.
+All primary and organizational models in NetBox support journaling. A journal is a collection of human-generated notes and comments about an object maintained for historical context. It supplements NetBox's change log to provide additional information about why changes have been made or to convey events which occur outside NetBox. Unlike the change log, in which records typically expire after a configurable period of time, journal entries persist for the life of their associated object.
 
 Each journal entry has a selectable kind (info, success, warning, or danger) and a user-populated `comments` field. Each entry automatically records the date, time, and associated user upon being created.

docs/features/search.md

@@ -0,0 +1,27 @@
+# Search
+
+## Global Search
+
+NetBox includes a powerful global search engine, providing a single convenient interface to search across its complex data model. Relevant fields on each model are indexed according to their precedence, so that the most relevant results are returned first. When objects are created or modified, the search index is updated immediately, ensuring real-time accuracy.
+
+When entering a search query, the user can choose a specific lookup type: exact match, partial match, etc. When a partial match is found, the matching portion of the applicable field value is included with each result so that the user can easily determine its relevance.
+
+Custom fields defined by NetBox administrators are also included in search results if configured with a search weight. Additionally, NetBox plugins can register their own custom models for inclusion alongside core models.
+
+## Saved Filters
+
+Each type of object in NetBox is accompanied by an extensive set of filters, each tied to a specific attribute, which enable the creation of complex queries. Often you'll find that certain queries are used routinely to apply some set of prescribed conditions to a query. Once a set of filters has been applied, NetBox offers the option to save it for future use.
+
+For example, suppose you often need to locate all planned devices of a certain type within a region. The applicable filters can be applied and then saved as custom named filter for reuse, such that
+
+```
+?status=planned&device_type_id=78&region_id=12
+```
+
+becomes
+
+```
+?filter=my-custom-filter
+```
+
+These saved filters can be used both within the UI and for API queries.

docs/getting-started/populating-data.md

@@ -20,12 +20,14 @@ To create a new object in NetBox, find the object type in the navigation menu an
 
 ## Bulk Import (CSV/YAML)
 
-NetBox supports the bulk import of new objects using CSV-formatted data. This method can be ideal for importing spreadsheet data, which is very easy to convert to CSV data. CSV data can be imported either as raw text using the form field, or by uploading a properly formatted CSV file.
+NetBox supports the bulk import of new objects, and updating of existing objects using CSV-formatted data. This method can be ideal for importing spreadsheet data, which is very easy to convert to CSV data. CSV data can be imported either as raw text using the form field, or by uploading a properly formatted CSV file.
 
 When viewing the CSV import form for an object type, you'll notice that the headers for the required columns have been pre-populated. Each form has a table beneath it titled "CSV Field Options," which lists _all_ supported columns for your reference. (Generally, these map to the fields you see in the corresponding creation form for individual objects.)
 
 <!-- TODO: Screenshot -->
 
+If an "id" field is added the data will be used to update existing records instead of importing new objects.
+
 Note that some models (namely device types and module types) do not support CSV import. Instead, they accept YAML-formatted data to facilitate the import of both the parent object as well as child components.
 
 ## Scripting

docs/installation/1-postgresql.md

@@ -2,8 +2,8 @@
 
 This section entails the installation and configuration of a local PostgreSQL database. If you already have a PostgreSQL database service in place, skip to [the next section](2-redis.md).
 
-!!! warning "PostgreSQL 10 or later required"
-    NetBox requires PostgreSQL 10 or later. Please note that MySQL and other relational databases are **not** supported.
+!!! warning "PostgreSQL 11 or later required"
+    NetBox requires PostgreSQL 11 or later. Please note that MySQL and other relational databases are **not** supported.
 
 ## Installation
 
@@ -35,7 +35,7 @@ sudo systemctl start postgresql
 sudo systemctl enable postgresql
 ```
 
-Before continuing, verify that you have installed PostgreSQL 10 or later:
+Before continuing, verify that you have installed PostgreSQL 11 or later:
 
 ```no-highlight
 psql -V

docs/installation/3-netbox.md

@@ -7,7 +7,7 @@ This section of the documentation discusses installing and configuring the NetBo
 Begin by installing all system packages required by NetBox and its dependencies.
 
 !!! warning "Python 3.8 or later required"
-    NetBox requires Python 3.8, 3.9, or 3.10.
+    NetBox requires Python 3.8, 3.9, 3.10 or 3.11.
 
 === "Ubuntu"
 

docs/installation/index.md

@@ -18,7 +18,7 @@ The following sections detail how to set up a new instance of NetBox:
 | Dependency | Minimum Version |
 |------------|-----------------|
 | Python     | 3.8             |
-| PostgreSQL | 10              |
+| PostgreSQL | 11              |
 | Redis      | 4.0             |
 
 Below is a simplified overview of the NetBox application stack for reference: