northd: Add ACL Sampling.

Introduce a new table called Sample where per-flow IPFIX configuration
can be specified.
Also, reference rows from such table from the ACL table to enable the
configuration of ACL sampling. If enabled, northd will add a sample
action to each ACL related logical flow.

Packets that hit stateful ACLs are sampled in different ways depending
whether they are initiating a new session or are just forwarded on an
existing (already allowed) session.  Two new columns ("sample_new" and
"sample_est") are added to the ACL table to allow for potentially
different sampling rates for the two cases.

Note: If an ACL has both sampling enabled and a label associated to it
then the label value overrides the observation point ID defined in the
sample configuration.  This is a side effect of the implementation
(observation point IDs are stored in conntrack in the same part of the
ct_label where ACL labels are also stored).  The two features
(sampling and ACL labels) serve however similar purposes so it's not
expected that they're both enabled together.

When sampling is enabled on an ACL additional logical flows are created
for that ACL (one for stateless ACLs and 3 for stateful ACLs) in the ACL
action stage of the logical pipeline.  These additional flows match on a
combination of conntrack state values and observation point id values
(either against a logical register or against the stored ct_label state)
in order to determine whether the packets hitting the ACLs must be
sampled or not.  This comes with a slight increase in the number of
logical flows and in the number of OpenFlow rules.  The number of
additional flows _does not_ depend on the original ACL match or action.

New --sample-new and --sample-est optional arguments are added to the
'ovn-nbctl acl-add' command to allow configuring these new types of
sampling for ACLs.

An example workflow of configuring ACL samples is:
  # Create Sampling_App mappings for ACL traffic types:
  ovn-nbctl create Sampling_App name="acl-new-traffic-sampling" \
                                id="42"
  ovn-nbctl create sampling_app name="acl-est-traffic-sampling" \
			        id="43"
  # Create two sample collectors, one that samples all packets (c1)
  # and another one that samples with a probability of 10% (c2):
  c1=$(ovn-nbctl create Sample_Collector name=c1 \
       probability=65535 set_id=1)
  c2=$(ovn-nbctl create Sample_Collector name=c2 \
       probability=6553 set_id=2)
  # Create two sample configurations (for new and for established
  # traffic):
  s1=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4301)
  s2=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4302)
  # Create an ingress ACL to allow IP traffic:
  ovn-nbctl --sample-new=$s1 --sample-est=$s2 acl-add ls \
            from-lport 1 "ip" allow-related

The config above will generate IPFIX samples with:
- 8 MSB of observation domain id set to 42 (Sampling_App
  "acl-new-traffic-sampling" config) and observation point id
  set to 4301 (Sample s1) for packets that create a new
  connection
- 8 MSB of observation domain id set to 43 (Sampling_app
  "acl-est-traffic-sampling" config) and observation point id
  set to 4302 (Sample s2) for packets that are part of an already
  existing connection

Note: in general, all generated IPFIX sample observation domain IDs are
built by ovn-controller in the following way:
The 8 MSB taken from the sample action's obs_domain_id and the last 24
LSB taken from the Southbound logical datapath tunnel_key (datapath ID).

Reported-at: https://issues.redhat.com/browse/FDP-305
Signed-off-by: Adrian Moreno <[email protected]>
Co-authored-by: Ales Musil <[email protected]>
Signed-off-by: Ales Musil <[email protected]>
Co-authored-by: Dumitru Ceara <[email protected]>
Signed-off-by: Dumitru Ceara <[email protected]>
---
V4:
- added explicit sampling stages
- reduced set_id max supported value
- added support for tiered "pass" ACLs
- improved system test + added tiered ACL system test
- added Ales as co-author for most of the above
V3:
- Addressed Ilya's comment:
  - Bumped NB schema version.
V2:
- Addressed Adrian's comments:
  - fixed up observation domain id comment in commit log.
  - store the obs_domain_id in the ct_label as an 8 bit value (add a
    test).
  - removed redundant check in build_acl_sample_label_action().
  - added missing space after ternary ":" operator.
  - documented limitation for sampling ACLs with action "pass".
  - documented sample_new behavior for stateless ACLs.
- Removed unused OVN_CT_SAMPLE_ID_SET_BIT and OVN_CT_SAMPLE_ID_SET.

NEWS

@@ -50,6 +50,9 @@ Post v24.03.0
   - The NB_Global.debug_drop_domain_id configured value is now overridden by
     the ID associated with the Sampling_App record created for drop sampling
     (Sampling_App.name configured to be "drop-sampling").
+  - Add support for ACL sampling through the new Sample_Collector and Sample
+    tables.  Sampling is supported for both traffic that creates new
+    connections and for traffic that is part of an existing connection.
 
 OVN v24.03.0 - 01 Mar 2024
 --------------------------

controller/lflow.h

@@ -67,17 +67,17 @@ struct uuid;
 
 /* Start of LOG_PIPELINE_LEN tables. */
 #define OFTABLE_LOG_INGRESS_PIPELINE      8
-#define OFTABLE_OUTPUT_LARGE_PKT_DETECT  37
-#define OFTABLE_OUTPUT_LARGE_PKT_PROCESS 38
-#define OFTABLE_REMOTE_OUTPUT            39
-#define OFTABLE_LOCAL_OUTPUT             40
-#define OFTABLE_CHECK_LOOPBACK           41
+#define OFTABLE_OUTPUT_LARGE_PKT_DETECT  40
+#define OFTABLE_OUTPUT_LARGE_PKT_PROCESS 41
+#define OFTABLE_REMOTE_OUTPUT            42
+#define OFTABLE_LOCAL_OUTPUT             43
+#define OFTABLE_CHECK_LOOPBACK           44
 
 /* Start of the OUTPUT section of the pipeline. */
 #define OFTABLE_OUTPUT_INIT OFTABLE_OUTPUT_LARGE_PKT_DETECT
 
 /* Start of LOG_PIPELINE_LEN tables. */
-#define OFTABLE_LOG_EGRESS_PIPELINE      42
+#define OFTABLE_LOG_EGRESS_PIPELINE      45
 #define OFTABLE_SAVE_INPORT              64
 #define OFTABLE_LOG_TO_PHY               65
 #define OFTABLE_MAC_BINDING              66

lib/logical-fields.c

@@ -175,6 +175,10 @@ ovn_init_symtab(struct shash *symtab)
                                     WR_CT_COMMIT);
     expr_symtab_add_subfield_scoped(symtab, "ct_label.label", NULL,
                                     "ct_label[96..127]", WR_CT_COMMIT);
+    expr_symtab_add_subfield_scoped(symtab, "ct_label.obs_point_id", NULL,
+                                    "ct_label[96..127]", WR_CT_COMMIT);
+    expr_symtab_add_subfield_scoped(symtab, "ct_label.obs_unused", NULL,
+                                    "ct_label[0..95]", WR_CT_COMMIT);
 
     expr_symtab_add_field(symtab, "ct_state", MFF_CT_STATE, NULL, false);
 

lib/ovn-util.h

@@ -308,7 +308,7 @@ BUILD_ASSERT_DECL(
 #define SCTP_ABORT_CHUNK_FLAG_T (1 << 0)
 
 /* The number of tables for the ingress and egress pipelines. */
-#define LOG_PIPELINE_LEN 29
+#define LOG_PIPELINE_LEN 30
 
 static inline uint32_t
 hash_add_in6_addr(uint32_t hash, const struct in6_addr *addr)