Merge remote-tracking branch 'origin/topic/timw/rst-linting'

* origin/topic/timw/rst-linting:
  Regenerate spicy docs
  Regenerate docs for rst linting fixes from scripts and bifs
  Fix uses of backticks
  Add pre-commit for linting rst, fix colon issues
  Use :file: directive for all log files in non-generated code

.pre-commit-config.yaml

@@ -7,3 +7,11 @@ repos:
     - id: ruff-format
     - id: ruff
       args: [--fix]
+
+- repo: https://github.com/pre-commit/pygrep-hooks
+  rev: v1.10.0
+  hooks:
+  - id: rst-directive-colons
+    stages: ["pre-commit"]
+  - id: rst-backticks
+    stages: ["pre-commit"]

cluster-setup.rst

@@ -177,7 +177,7 @@ Offloading and ethtool tuning
 While not specific to AF_PACKET, it is recommended to disable any offloading
 features provided by the network card or Linux networking stack when running
 Zeek. This allows to see network packets as they arrive on the wire.
-See this `blog post <https://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html`>_
+See this `blog post <https://blog.securityonion.net/2011/10/when-is-full-packet-capture-not-full.html>`_
 for more background
 
 Toggling these features can be done with the ``ethtool -K`` command, for example::
@@ -600,4 +600,3 @@ same packets multiple times with different tools.
 
        This issue has been fixed in all stable kernels for at least 5 years.
        You're unlikely to be affected.
-

customizations.rst

@@ -286,14 +286,14 @@ package helps you with that.
 Long Connections
 ----------------
 
-Zeek logs connection entries into the ``conn.log`` only upon termination
+Zeek logs connection entries into the :file:`conn.log` only upon termination
 or due to expiration of inactivity timeouts. Depending on the protocol and
 chosen timeout values this can significantly delay the appearance of a log
 entry for a given connection. The delay may be up to an hour for lingering
 SSH connections or connections where the final FIN or RST packets were missed.
 
-The `zeek-long-connections`_ package alleviates this by creating a ``conn_long.log``
-log with the same format as ``conn.log``, but containing entries for connections
+The `zeek-long-connections`_ package alleviates this by creating a :file:`conn_long.log`
+log with the same format as :file:`conn.log`, but containing entries for connections
 that have been existing for configurable intervals.
 By default, the first entry for a connection is logged after 10mins. Depending on
 the environment, this can be lowered as even a 10 minute delay may be significant

devel/plugins.rst

@@ -449,7 +449,7 @@ If your plugin isn't loading as expected, Zeek's debugging facilities
 can help illuminate what's going on. To enable, recompile Zeek
 with debugging support (``./configure --enable-debug``), and
 afterwards rebuild your plugin as well. If you then run Zeek with ``-B
-plugins``, it will produce a file ``debug.log`` that records details
+plugins``, it will produce a file :file:`debug.log` that records details
 about the process for searching, loading, and activating plugins.
 
 To generate your own debugging output from inside your plugin, you can
@@ -463,7 +463,7 @@ your plugin's debugging output with ``-B plugin-<name>``, where
 ``Configure()`` method, yet with the namespace-separator ``::``
 replaced with a simple dash. Example: If the plugin is called
 ``Demo::Rot13``, use ``-B plugin-Demo-Rot13``. As usual, the debugging
-output will be recorded to ``debug.log`` if Zeek's compiled in debug
+output will be recorded to :file:`debug.log` if Zeek's compiled in debug
 mode.
 
 Building Plugins Statically

devel/spicy/autogen/init-bare.zeek

@@ -23,7 +23,7 @@ export {
 # doc-options-end
 
 # doc-types-start
-    ## Result type for `Spicy::resource_usage()`. The values reflect resource
+    ## Result type for ``Spicy::resource_usage()``. The values reflect resource
     ## usage as reported by the Spicy runtime system.
     type ResourceUsage: record {
         user_time : interval;           ##< user CPU time of the Zeek process

devel/spicy/reference.rst

@@ -305,7 +305,7 @@ File analyzers support the following properties:
             "content sniffing" (i.e., similar to libmagic), and
             usually not by protocol-level headers (e.g., *not* through
             HTTP's ``Content-Type`` header). If in doubt, examine
-            ``files.log`` for what it records as a file's type.
+            :file:`files.log` for what it records as a file's type.
 
     ``replaces ANALYZER_NAME``
         Disables an existing file analyzer that Zeek already provides
@@ -942,7 +942,7 @@ membership in a ``set`` like this:
 
 .. code-block:: zeek
 
-   # Zeek module `MyModule`
+   # Zeek module MyModule
    option my_set: set[count] = { 1, 2, 3 };
 
 .. code-block:: spicy
@@ -958,7 +958,7 @@ functions for conversion. Example accessing a record's field:
 
 .. code-block:: zeek
 
-   # Zeek module `MyModule`
+   # Zeek module MyModule
    option my_record: record {
        a: count &default = 42;
        b: string &default = "foo";
@@ -1103,7 +1103,7 @@ see what it's doing at runtime. You'll need a debug version of Zeek
 for that, as well as a small trace with traffic that you expect your
 analyzer to process. Run Zeek with ``-B dpd`` (or ``-B file_analysis``
 if you're debugging a file analyzer) on your trace to record the
-analyzer activity into ``debug.log``. For example, with the same HTTP
+analyzer activity into :file:`debug.log`. For example, with the same HTTP
 example, we get:
 
 .. code-block:: text
@@ -1189,4 +1189,3 @@ their full Zeek-side values::
                   [2] uri: string        = /index.html
                   [3] version: string    = 1.0
     [...]
-

devel/spicy/tutorial.rst

@@ -208,7 +208,7 @@ the corresponding value of ``is_read``. Let's try it with a new
     # zeek -r tftp_rrq.pcap tftp.hlto tftp.zeek
     TFTP read request, [orig_h=192.168.0.253, orig_p=50618/udp, resp_h=192.168.0.10, resp_p=69/udp], T, rfc1350.txt, octet
 
-If we look at the ``conn.log`` that Zeek produces during this run, we
+If we look at the :file:`conn.log` that Zeek produces during this run, we
 will see that the ``service`` field is not filled in yet. That's
 because our analyzer does not yet confirm to Zeek that it has been
 successful in parsing the content. To do that, we can call a library
@@ -333,7 +333,7 @@ analyzers' events, and collect and correlate their activity as
 desired. We have created such :download:`a script for TFTP
 <autogen/tftp.zeek>`, based on the events that our Spicy analyzer
 generates. Once we add that to the Zeek command line, we will see a
-new ``tftp.log``:
+new :file:`tftp.log`:
 
 .. code::
 
@@ -346,7 +346,7 @@ new ``tftp.log``:
 The TFTP script also labels the second session as TFTP data by
 adding a corresponding entry to the ``service`` field inside the
 Zeek-side connection record. With that, we are now seeing this in
-``conn.log``:
+:file:`conn.log`:
 
 .. code::
 

ext/spicy-pygments.py

@@ -1,4 +1,4 @@
-# Copyright (c) 2020-2023 by the Zeek Project. See LICENSE for details.
+# Copyright (c) 2020-now by the Zeek Project. See LICENSE for details.
 
 from pygments.lexer import RegexLexer, bygroups, include, words
 from pygments.token import (

frameworks/file-analysis.rst

@@ -84,12 +84,12 @@ File Type Identification
 Zeek ships with its own library of content signatures to determine the type of a
 file, conveyed as MIME types in the :zeek:see:`file_sniff` event. You can find
 those signatures in the Zeek distribution's ``scripts/base/frameworks/files/magic/``
-directory. (Despite the name, Zeek does `not` rely on libmagic for content analysis.)
+directory. (Despite the name, Zeek does *not* rely on libmagic for content analysis.)
 
 Adding Analysis
 ===============
 
-Zeek supports customized file analysis via `file analyzers` that users can
+Zeek supports customized file analysis via *file analyzers* that users can
 attach to observed files. You can attach analyzers selectively to individual
 files, or register them for auto-attachment under certain conditions. Once
 attached, file analyzers start receiving the contents of files as Zeek parses
@@ -139,7 +139,7 @@ for additional APIs and data structures.
 
 Regardless of which file analyzers end up acting on a file, general
 information about the file (e.g. size, time of last data transferred,
-MIME type, etc.) is logged in ``files.log``.
+MIME type, etc.) is logged in :file:`files.log`.
 
 Protocol-specific state
 -----------------------
@@ -207,8 +207,8 @@ customized analysis. Since observed files can be very large, Zeek cannot buffer
 these files and provide their entire content to the script layer once
 complete. Instead, the ``FileDataEvent`` analyzer reflects the incremental
 nature of file content as Zeek observes it, and supports two types of events to
-allow you to process it: user-provided `stream events` receive new file content
-as supplied by connection-oriented protocols, while `chunk events` receive
+allow you to process it: user-provided *stream events* receive new file content
+as supplied by connection-oriented protocols, while *chunk events* receive
 observed data as provided by protocols that do not feature stream semantics.
 
 The following example manually computes the SHA256 hash of each observed file by

frameworks/input.rst

@@ -318,7 +318,7 @@ Broken input data
 -----------------
 
 The input framework notifies you of problems during data ingestion in two ways.
-First, reporter messages, ending up in reporter.log, indicate the type of
+First, reporter messages, ending up in :file:`reporter.log`, indicate the type of
 problem and the file in which the problem occurred::
 
   #fields ts      level   message location

frameworks/logging.rst

@@ -261,7 +261,7 @@ Disable a Stream
 ----------------
 
 One way to “turn off” a log is to completely disable the stream. For example,
-the following example will prevent the conn.log from being written:
+the following example will prevent the :file:`conn.log` from being written:
 
 .. code-block:: zeek
 

frameworks/management.rst

@@ -764,7 +764,7 @@ Retrieving configurations
 -------------------------
 
 The client's ``get-config`` command lets you retrieve staged and deployed
-configurations from the controller, in JSON or `.ini` form. This is helpful for
+configurations from the controller, in JSON or :file:`.ini` form. This is helpful for
 examining the differences between the two. Following the successful deployment
 shown above, we have:
 

frameworks/netcontrol.rst

@@ -141,7 +141,7 @@ sees being established:
 
 Running this script on a file containing one connection will cause the debug
 plugin to print one line to the standard output, which contains information
-about the rule that was added. It will also cause creation of `netcontrol.log`,
+about the rule that was added. It will also cause creation of :file:`netcontrol.log`,
 which contains information about all actions that are taken by NetControl:
 
 .. code-block:: console
@@ -166,7 +166,7 @@ which contains information about all actions that are taken by NetControl:
    1398529018.678276 2       NetControl::RULE        ADD     NetControl::SUCCEEDED   NetControl::DROP        NetControl::FORWARD     NetControl::CONNECTION  192.168.18.50/56981<->74.125.239.97/443 -       -       0       20.000000       -       Debug-All
    #close    2018-12-14-18-50-53
 
-In our case, `netcontrol.log` contains several :zeek:see:`NetControl::MESSAGE`
+In our case, :file:`netcontrol.log` contains several :zeek:see:`NetControl::MESSAGE`
 entries, which show that the debug plugin has been initialized and added.
 Afterwards, there are two :zeek:see:`NetControl::RULE` entries; the first shows
 that the addition of a rule has been requested (state is
@@ -175,8 +175,8 @@ successfully added (the state is :zeek:see:`NetControl::SUCCEEDED`). The
 remainder of the log line gives more information about the added rule, which in
 our case applies to a specific 5-tuple.
 
-In addition to the netcontrol.log, the drop commands also create a second,
-additional log called `netcontrol_drop.log`. This log file is much more succinct and
+In addition to the :file:`netcontrol.log`, the drop commands also create a second,
+additional log called :file:`netcontrol_drop.log`. This log file is much more succinct and
 only contains information that is specific to drops that are enacted by
 NetControl:
 
@@ -259,7 +259,7 @@ the :zeek:see:`Notice::ACTION_DROP` action of the notice framework:
    #close    2018-12-14-18-50-55
 
 Using the :zeek:see:`Notice::ACTION_DROP` action of the notice framework also
-will cause the `dropped` column in `notice.log` to be set to true each time that
+will cause the ``dropped`` column in :file:`notice.log` to be set to true each time that
 the NetControl framework enacts a block:
 
 .. code-block:: console
@@ -540,7 +540,7 @@ for 10 minutes and monitored for 1 hour. If the address reappears after the
 first 10 minutes, it is blocked for 1 hour and then monitored for 24 hours, etc.
 
 Catch and release adds its own new logfile in addition to the already existing
-ones (netcontrol_catch_release.log):
+ones (:file:`netcontrol_catch_release.log`):
 
 .. code-block:: console
 
@@ -631,7 +631,7 @@ use the plugin, we first had to instantiate it by calling
 calling :zeek:see:`NetControl::activate`.
 
 As we already hinted before, NetControl supports having several plugins that are
-active at the same time. The second argument to the `NetControl::activate`
+active at the same time. The second argument to the :zeek:see:`NetControl::activate`
 function is the priority of the backend that was just added. Each rule is sent
 to all plugins in order, from highest priority to lowest priority. The backend
 can then choose if it accepts the rule and pushes it out to the hardware that it
@@ -646,7 +646,7 @@ network with two OpenFlow switches. The first switch forwards packets from the
 network to the external world, the second switch sits in front of your Zeek
 cluster to provide packet shunting. In this case, you can add two OpenFlow
 backends to NetControl. When you create the instances using
-:zeek:see:`NetControl::create_openflow`, you set the `monitor` and `forward`
+:zeek:see:`NetControl::create_openflow`, you set the ``monitor`` and ``forward``
 attributes of the configuration in :zeek:see:`NetControl::OfConfig`
 appropriately. Afterwards, one of the backends will only accept rules for the
 monitor path; the other backend will only accept rules for the forward path.
@@ -660,7 +660,7 @@ rule if the rule matches the subnet that the specific switch is responsible for.
 To give an example, the following script adds two backends to NetControl. One
 backend is the NetControl debug backend, which just outputs the rules to the
 console. The second backend is an OpenFlow backend, which uses the OpenFlow
-debug mode that outputs the openflow rules to openflow.log. The OpenFlow
+debug mode that outputs the openflow rules to :file:`openflow.log`. The OpenFlow
 backend uses a predicate function to only accept rules with a source address in
 the 192.168.17.0/24 network; all other rules will be passed on to the debug
 plugin. We manually block a few addresses in the
@@ -680,7 +680,7 @@ plugin. We manually block a few addresses in the
 
 As you can see, only the single block affecting the 192.168.17.0/24 network is
 output to the command line. The other two lines are handled by the OpenFlow
-plugin. We can verify this by looking at netcontrol.log. The plugin column shows
+plugin. We can verify this by looking at :file:`netcontrol.log`. The plugin column shows
 which plugin handled a rule and reveals that two rules were handled by OpenFlow:
 
 .. code-block:: console
@@ -707,7 +707,7 @@ which plugin handled a rule and reveals that two rules were handled by OpenFlow:
    1544813458.913148 4       NetControl::RULE        ADD     NetControl::SUCCEEDED   NetControl::DROP        NetControl::FORWARD     NetControl::ADDRESS     192.168.18.2/32 -       -       0       60.000000       -       Openflow-Log-42
    #close    2018-12-14-18-50-58
 
-Furthermore, openflow.log also shows the two added rules, converted to OpenFlow
+Furthermore, :file:`openflow.log` also shows the two added rules, converted to OpenFlow
 flow mods:
 
 .. code-block:: console
@@ -758,13 +758,13 @@ communicate with the Switch.
     NetControl and OpenFlow architecture (click to enlarge).
 
 The Python scripts that are used to interface with the available NetControl
-plugins are contained in the `zeek-netcontrol` repository (`github link <https://github.com/zeek/zeek-netcontrol>`_).
+plugins are contained in the ``zeek-netcontrol`` repository (`github link <https://github.com/zeek/zeek-netcontrol>`_).
 The repository contains scripts for the OpenFlow as well as the acld plugin.
 Furthermore, it contains a script for the broker plugin which can be used to
 call configurable command-line programs when used with the broker plugin.
 
 The repository also contains documentation on how to install these connectors.
-The `netcontrol` directory contains an API that allows you to write your own
+The ``netcontrol`` directory contains an API that allows you to write your own
 connectors to the broker plugin.
 
 Writing plugins

frameworks/notice.rst

@@ -359,7 +359,7 @@ Configuring Notice Emails
 If :zeek:see:`Notice::mail_dest` is set, notices with an associated
 e-mail action will be sent to that address. For additional
 customization, users can use the :zeek:see:`Notice::policy` hook to
-modify the `email_dest` field. The following example would result in 3
+modify the ``email_dest`` field. The following example would result in three
 separate e-mails:
 
 .. code-block:: zeek
@@ -375,7 +375,7 @@ separate e-mails:
 
 You can also use :zeek:see:`Notice::policy` hooks to add extra information to
 emails. The :zeek:see:`Notice::Info` record contains a vector of strings named
-`$email_body_sections` which Zeek will include verbatim when sending email.
+``email_body_sections`` which Zeek will include verbatim when sending email.
 An example of including some information from an HTTP request is included below.
 
 .. code-block:: zeek

frameworks/signatures.rst

@@ -331,7 +331,7 @@ two actions defined, ``event`` and ``enable``.
     .. note::
 
       Matches for signatures that use custom events do not appear
-      in ``signatures.log``.
+      in :file:`signatures.log`.
 
 
 ``enable <string>``
@@ -484,4 +484,3 @@ the two systems are just too different. We therefore stopped maintaining
 the ``snort2bro`` script, and there are now many newer Snort options
 which it doesn't support. The script is now no longer part of the Zeek
 distribution.
-

frameworks/telemetry.rst

@@ -87,7 +87,7 @@ centralize, or aggregate metrics across the cluster. Instead, it adds the name
 of the node a particular metric originated from at collection time, leaving any
 aggregation to post-processing where desired.
 
-.. note:
+.. note::
 
    This is a departure from the design in earlier versions of Zeek, which could
    (either by default, or after activation) centralize metrics in the cluster's
@@ -107,8 +107,8 @@ them in turn.
 Zeek Logs
 ---------
 
-Zeek can export current metrics continuously via `telemetry.log` and
-`telemetry_histogram.log`. It does not do so by default. To enable, load the
+Zeek can export current metrics continuously via :file:`telemetry.log` and
+:file:`telemetry_histogram.log`. It does not do so by default. To enable, load the
 policy script ``frameworks/telemetry/log`` on the command line, or via
 ``local.zeek``.
 
@@ -119,7 +119,7 @@ cluster node the metric originated from.
 By default, Zeek reports current telemetry every 60 seconds, as defined by the
 :zeek:see:`Telemetry::log_interval`, which you're free to adjust.
 
-Also, by default only metrics with the `prefix` (namespace) ``zeek`` and
+Also, by default only metrics with the ``prefix`` (namespace) ``zeek`` and
 ``process`` are included in above logs. If you add new metrics with your own
 prefix and expect these to be included, redefine the
 :zeek:see:`Telemetry::log_prefixes` option::
@@ -404,7 +404,7 @@ representation becomes rather verbose.
 To work with histogram data, Prometheus provides specialized query functions.
 For example `histogram_quantile()`_.
 
-Note, when using data from `conn.log` and post-processing, a proper
+Note, when using data from :file:`conn.log` and post-processing, a proper
 histogram of connection durations can be calculated and possibly preferred.
 The above example is meant for demonstration purposes. Histograms may be
 primarily be useful for Zeek operational metrics such as processing times