(DOCSP-43962) [Kafka] Kafka connector nested components (#167) (#171)

* (DOCSP-43962) Unnested 1 example in tip. * (DOCSP-43962) Removing the subjunctive should * (DOCSP-42962) Unnested 1 important admonition in table. * (DOCSP-42962) Typo fix. * (DOCSP-42962) Unnested 1 important admonition in table. * (DOCSP-42962) Unnested 1 tip in table. * (DOCSP-42962) Fixing indentation error and unnesting 1 important admonition in table. * (DOCSP-42962) Spacing. * (DOCSP-42962) Build errors. * (DOCSP-42962) Unnested 1 example and 1 note in table. * (DOCSP-42962) Unnested 2 examples and 1 note in include. * (DOCSP-42962) Replace quotation marks with monospace. * (DOCSP-42962) Unnested 2 notes in table. * (DOCSP-42962) consistency. * (DOCSP-42962) Edited for consistency and unnested 2 tips. * (DOCSP-42962) Removing extra line breaks. * (DOCSP-42962) Unnested 2 important admonitions in table. * (DOCSP-42962) Fix rendering error. * (DOCSP-42962) Fixing rendering issues. * (DOCSP-42962) Unnested 2 notes and 1 example in table. * (DOCSP-42962) Unnested 3 important admonitions * (DOCSP-42962) Spacing fix. * (DOCSP-42962) Unnested 1 example and 3 tips. * (DOCSP-42962) Unnested 1 note, 2 tips, 1 important admonition. * (DOCSP-42962) Spacing. * (DOCSP-42962) Spacing. * (DOCSP-42962) Normalizing spacing and adding reference to the Data Formats page. * (DOCSP-42962) Unnested 3 tips, 1 example and 1 important admonition in table. * (DOCSP-42962) Regularizing usage of pipeline operator line breaks. * (DOCSP-42962) Spacing. * (DOCSP-42962) Spacing. * (DOCSP-42962) Normalizing spacing across pages. * Update source/source-connector/configuration-properties/kafka-topic.txt * Update source/source-connector/configuration-properties/kafka-topic.txt --------- Co-authored-by: carriecwk <[email protected]>
mongodb · Oct 4, 2024 · 12ef322 · 12ef322
1 parent 007cc76
commit 12ef322
Show file tree

Hide file tree

Showing 20 changed files with 258 additions and 276 deletions.
diff --git a/source/includes/copy-existing-admonition.rst b/source/includes/copy-existing-admonition.rst
@@ -1,7 +1,5 @@
-.. note:: Data Copy Can Produce Duplicate Events
-
-    If any system changes the data in the database while the source connector
-    converts existing data from it, MongoDB may produce duplicate change
-    stream events to reflect the latest changes.  Since the change stream
-    events on which the data copy relies are idempotent, the copied data is
-    eventually consistent.
+If any system changes the data in the database while the source connector
+converts existing data from it, MongoDB may produce duplicate change
+stream events to reflect the latest changes.  Since the change stream
+events on which the data copy relies are idempotent, the copied data is
+eventually consistent.
diff --git a/source/includes/externalize-secrets.rst b/source/includes/externalize-secrets.rst
@@ -1,7 +1,5 @@
-.. important:: Avoid Exposing Your Authentication Credentials
-
-   To avoid exposing your authentication credentials in your
-   ``connection.uri`` setting, use a
-   `ConfigProvider <https://docs.confluent.io/current/connect/security.html#externalizing-secrets>`__
-   and set the appropriate configuration parameters.
+:gold:`IMPORTANT:` To avoid exposing your authentication credentials in your
+``connection.uri`` setting, use a 
+`ConfigProvider <https://docs.confluent.io/current/connect/security.html#externalizing-secrets>`__
+and set the appropriate configuration parameters.
 
diff --git a/source/introduction/data-formats.txt b/source/introduction/data-formats.txt
@@ -1,3 +1,5 @@
+.. _kafka-data-formats:
+
 ============
 Data Formats
 ============

diff --git a/source/sink-connector/configuration-properties/connector-message.txt b/source/sink-connector/configuration-properties/connector-message.txt
@@ -75,8 +75,9 @@ Settings
          When set to ``true``, the default value, the
          connector writes a batch of records as an ordered bulk write
          operation.
-       | To learn more about bulk write operations, see the :ref:`Write
-         Model Strategies page <sink-connector-bulk-write-ops>`.
+       |
+       | To learn more about bulk write operations, see 
+         :ref:`Bulk Write Operations <sink-connector-bulk-write-ops>`.
        |
        | **Default**: ``true``
        | **Accepted Values**: ``true`` or ``false``
@@ -111,15 +112,13 @@ Settings
        | The maximum number of tasks to create for this connector. The
          connector may create fewer than the maximum tasks specified if it
          cannot handle the level of parallelism you specify.
-
-       .. important:: Multiple Tasks May Process Messages Out of Order
-
-          If you specify a value greater than ``1``, the connector enables
-          parallel processing of the tasks. If your topic has multiple
-          partition logs, which enables the connector to read from the
-          topic in parallel, the tasks may process the messages out of
-          order.
-
+       |
+       | :gold:`IMPORTANT:` If you specify a value greater than ``1``, 
+         the connector enables parallel processing of the tasks. 
+         If your topic has multiple partition logs, which enables 
+         the connector to read from the topic in parallel, 
+         the tasks may process the messages out of order.
+       |
        | **Default**: ``1``
        | **Accepted Values**: An integer
 

diff --git a/source/sink-connector/configuration-properties/error-handling.txt b/source/sink-connector/configuration-properties/error-handling.txt
@@ -48,19 +48,21 @@ Settings
        | Whether to continue processing messages if the connector encounters
          an error. Allows the connector to override  the ``errors.tolerance``
          Kafka cluster setting.
+       |
        | When set to ``none``, the connector reports any error and
          blocks further processing of the rest of the messages.
+       |
        | When set to ``all``, the connector ignores any problematic messages.
+       |
        | When set to ``data``, the connector tolerates only data errors and
          fails on all other errors.
+       |
        | To learn more about error handling strategies, see the
          :ref:`<kafka-sink-handle-errors>` page.
-
-       .. note::
-
-          This property overrides the `errors.tolerance <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-tolerance>`__
-          property of the Connect Framework.
-
+       |
+       | This property overrides the `errors.tolerance <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-tolerance>`__
+       | property of the Connect Framework.
+       |
        | **Default:** Inherits the value from the ``errors.tolerance``
          setting.
        | **Accepted Values**: ``"none"`` or ``"all"``
@@ -73,16 +75,14 @@ Settings
          failed operations to the log file. The connector classifies
          errors as "tolerated" or "not tolerated" using the
          ``errors.tolerance`` or ``mongo.errors.tolerance`` settings.
-
+       |
        | When set to ``true``, the connector logs both "tolerated" and
          "not tolerated" errors.
        | When set to ``false``, the connector logs "not tolerated" errors.
-
-       .. note::
-
-          This property overrides the `errors.log.enable <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-log-enable>`__
-          property of the Connect Framework.
-
+       | 
+       | This property overrides the `errors.log.enable <https://docs.confluent.io/platform/current/installation/configuration/connect/sink-connect-configs.html#errors-log-enable>`__
+       | property of the Connect Framework.
+       |
        | **Default:** ``false``
        | **Accepted Values**: ``true`` or ``false``
 
@@ -104,7 +104,8 @@ Settings
        | Name of topic to use as the dead letter queue. If blank, the
          connector does not send any invalid messages to the dead letter
          queue.
-       | For more information about the dead letter queue, see the
+       |
+       | To learn more about the dead letter queue, see the
          :ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
        |
        | **Default:** ``""``
@@ -116,11 +117,13 @@ Settings
        | **Description:**
        | Whether the connector should include context headers when it
          writes messages to the dead letter queue.
+       |
        | To learn more about the dead letter queue, see the
          :ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
+       |
        | To learn about the exceptions the connector defines and
-         reports through context headers, see the
-         :ref:`<sink-configuration-error-handling-dlq-errors>` section.
+         reports through context headers, see
+         :ref:`<sink-configuration-error-handling-dlq-errors>`.
        |
        | **Default:** ``false``
        | **Accepted Values**: ``true`` or ``false``
@@ -132,7 +135,8 @@ Settings
        | The number of nodes on which to replicate the dead letter queue
          topic. If you are running a single-node Kafka cluster, you must
          set this to ``1``.
-       | For more information about the dead letter queue, see the
+       |
+       | To learn more about the dead letter queue, see the
          :ref:`Dead Letter Queue Configuration Example <sink-dead-letter-queue-configuration-example>`.
        |
        | **Default:** ``3``

diff --git a/source/sink-connector/configuration-properties/id-strategy.txt b/source/sink-connector/configuration-properties/id-strategy.txt
@@ -76,8 +76,9 @@ Settings
        |
        | **Description:**
        | Whether the connector should delete documents when the key value
-         matches a document in MongoDB and the value field is null. This 
-         setting applies when you specify an id generation strategy that 
+         matches a document in MongoDB and the value field is null. 
+       |
+       | This setting applies when you specify an id generation strategy that 
          operates on the key document such as ``FullKeyStrategy``, 
          ``PartialKeyStrategy``, and ``ProvidedInKeyStrategy``.
        |

diff --git a/source/sink-connector/configuration-properties/kafka-topic.txt b/source/sink-connector/configuration-properties/kafka-topic.txt
@@ -42,10 +42,8 @@ Settings
        | **Description:**
        | A list of Kafka topics that the sink connector watches.
 
-       .. note::
-
-          You can define either the ``topics`` or the ``topics.regex``
-          setting, but not both.
+       You can define either the ``topics`` or the ``topics.regex``
+       setting, but not both.
 
        | **Accepted Values**: A comma-separated list of valid Kafka topics
 
@@ -58,20 +56,16 @@ Settings
        | A regular expression that matches the Kafka topics that the sink
          connector watches.
 
-       .. example::
-
-          .. code-block:: properties
-
-             topics.regex=activity\\.\\w+\\.clicks$
+       For example, the following regex matches topic names such as 
+       "activity.landing.clicks" and "activity.support.clicks". 
+       It does not match the topic names "activity.landing.views" and "activity.clicks".
 
-          This regex matches topic names such as "activity.landing.clicks"
-          and "activity.support.clicks". It does not match the topic names 
-          "activity.landing.views" and "activity.clicks".
+       .. code-block:: properties
 
-       .. note::
+          topics.regex=activity\\.\\w+\\.clicks$
 
-          You can define either the ``topics`` or the ``topics.regex``
-          setting, but not both.
+       You can define either the ``topics`` or the ``topics.regex``
+       setting, but not both.
 
        | **Accepted Values**: A valid regular expression pattern using ``java.util.regex.Pattern``.
 

diff --git a/source/sink-connector/configuration-properties/mongodb-connection.txt b/source/sink-connector/configuration-properties/mongodb-connection.txt
@@ -68,6 +68,7 @@ Settings
        | When set to ``true``, if the connector calls a command on your
          MongoDB instance that's deprecated in the declared {+stable-api+}
          version, it raises an exception.
+       |
        | You can set the API version with the ``server.api.version``
          configuration option. For more information on the {+stable-api+}, see
          the MongoDB manual entry on the
@@ -83,6 +84,7 @@ Settings
        | When set to ``true``, if the connector calls a command on your
          MongoDB instance that's not covered in the declared {+stable-api+}
          version, it raises an exception.
+       |
        | You can set the API version with the ``server.api.version``
          configuration option. For more information on the {+stable-api+}, see
          the MongoDB manual entry on the

diff --git a/source/sink-connector/configuration-properties/mongodb-namespace.txt b/source/sink-connector/configuration-properties/mongodb-namespace.txt
@@ -43,14 +43,12 @@ Settings
          database or collection in which to sink the data. The default
          ``DefaultNamespaceMapper`` uses values specified in the
          ``database`` and ``collection`` properties.
-
-       .. seealso::
-
-          The connector includes an alternative class for specifying the
-          database and collection called ``FieldPathNamespaceMapper``. See
-          the :ref:`FieldPathNamespaceMapper settings <fieldpathnamespacemapper-settings>`
-          for more information.
-
+       |
+       | The connector includes an alternative class for specifying the
+       | database and collection called ``FieldPathNamespaceMapper``. See
+       | :ref:`FieldPathNamespaceMapper Settings <fieldpathnamespacemapper-settings>`
+       | for more information.
+       |
        | **Default**:
 
        .. code-block:: none
@@ -149,10 +147,12 @@ You can use the following settings to customize the behavior of the
        | **Description**:
        | Whether to throw an exception when either the document is missing the
          mapped field or it has an invalid BSON type.
+       |
        | When set to ``true``, the connector does not process documents
          missing the mapped field or that contain an invalid BSON type.
          The connector may halt or skip processing depending on the related
          error-handling configuration settings.
+       |
        | When set to ``false``, if a document is missing the mapped field or
          if it has an invalid BSON type, the connector defaults to
          writing to the specified ``database`` and ``collection`` settings.

diff --git a/source/sink-connector/configuration-properties/post-processors.txt b/source/sink-connector/configuration-properties/post-processors.txt
@@ -40,13 +40,10 @@ Settings
        | **Description:**
        | A list of post-processor classes the connector should apply to
          process the data before saving it to MongoDB.
-
-       .. seealso::
-
-          For more information on post-processors and examples of
-          their usage, see the section on
-          :doc:`Post-processors </sink-connector/fundamentals/post-processors>`.
-
+       |
+       | To learn more about post-processors and see examples of
+       | their usage, see
+       | :doc:`Sink Connector Post Processors </sink-connector/fundamentals/post-processors>`.
        |
        | **Default**:
 
@@ -129,12 +126,9 @@ Settings
        | **Description:**
        | The class that specifies the ``WriteModelStrategy`` the connector should
          use for :manual:`Bulk Writes </core/bulk-write-operations/index.html>`.
-
-       .. seealso::
-
-         For information on how to create your own strategy, see
-         :ref:`<kafka-sink-write-model-create-custom-strategy>`.
-
+       | 
+       | To learn more about how to create your own strategy, see
+       | :ref:`<kafka-sink-write-model-create-custom-strategy>`.
        |
        | **Default**:
 

diff --git a/source/sink-connector/configuration-properties/time-series.txt b/source/sink-connector/configuration-properties/time-series.txt
@@ -57,9 +57,11 @@ Settings
        | The date format pattern the connector should use to convert the
          source data contained in the field specified by the
          ``timeseries.timefield`` setting.
-         The connector passes the date format pattern to the Java
+       |
+       | The connector passes the date format pattern to the Java
          `DateTimeFormatter.ofPattern(pattern, locale) <https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html#ofPattern-java.lang.String-java.util.Locale->`__
          method to perform date and time conversions on the time field.
+       |
        | If the date value from the source data only contains date information,
          the connector sets the time information to the start of the specified
          day. If the date value does not contain the timezone offset, the
@@ -79,6 +81,7 @@ Settings
        | **Description:**
        | Whether to convert the data in the field into the BSON ``Date``
          format.
+       |
        | When set to ``true``, the connector uses the milliseconds
          after epoch and discards fractional parts if the value is
          a number. If the value is a string, the connector uses the
@@ -99,8 +102,9 @@ Settings
        |
        | **Description:**
        | Which ``DateTimeFormatter`` locale language tag to use with the date
-         format pattern (e.g. ``"en-US"``). For more information on
-         locales, see the Java SE documentation of `Locale <https://docs.oracle.com/javase/8/docs/api/java/util/Locale.html>`__.
+         format pattern (e.g. ``"en-US"``). 
+       |
+       | To learn more about locales, see the Java SE documentation of `Locale <https://docs.oracle.com/javase/8/docs/api/java/util/Locale.html>`__.
        |
        | **Default**: ``ROOT``
        | **Accepted Values**: A valid ``Locale`` language tag format
@@ -111,12 +115,10 @@ Settings
        | **Description:**
        | Which top-level field to read from the source data to describe
          a group of related time series documents.
-
-       .. important::
-
-          This field must not be the ``_id`` field nor the field you specified
-          in the ``timeseries.timefield`` setting.
-
+       | 
+       | :gold:`IMPORTANT:` This field must not be the ``_id`` field nor the field you specified
+         in the ``timeseries.timefield`` setting.
+       | 
        | **Default**: ``""``
        | **Accepted Values**: An empty string or the name of a field
          that contains any BSON type except ``BsonArray``.
@@ -128,8 +130,9 @@ Settings
        | The number of seconds MongoDB should wait before automatically
          removing the time series collection data. The connector disables
          timed expiry when the setting value is less than ``1``.
-         For more information on this collection setting, see the MongoDB
-         Server Manual page on :manual:`Automatic Removal for Time Series Collections </core/timeseries/timeseries-automatic-removal/>`.
+       |
+       | To learn more, see :manual:`Set up Automatic Removal for Time Series Collections </core/timeseries/timeseries-automatic-removal/>`
+         in the MongoDB manual. 
        |
        | **Default**: ``0``
        | **Accepted Values**: An integer
@@ -140,9 +143,10 @@ Settings
        |
        | **Description:**
        | The expected interval between subsequent measurements of your
-         source data. For more information on this setting, see the
-         MongoDB server manual page on :manual:`Granularity for Time
-         Series Data </core/timeseries/timeseries-granularity/>`.
+         source data. 
+       |
+       | To learn more, see :manual:`Set Granularity for Time Series Data
+         </core/timeseries/timeseries-granularity/>` in the MongoDB manual. 
        |
        | *Optional*
        | **Default**: ``""``

diff --git a/source/sink-connector/configuration-properties/topic-override.txt b/source/sink-connector/configuration-properties/topic-override.txt
@@ -40,19 +40,15 @@ Settings
        | **Description:**
        | Specify a topic and property name to override the corresponding
          global or default property setting.
-
-       .. example::
-
-          The ``topic.override.foo.collection=bar`` setting instructs the
-          sink connector to store data from the ``foo`` topic in the ``bar``
-          collection.
-
-       .. note::
-
-          You can specify any valid configuration setting in the
-          ``<propertyName>`` segment on a per-topic basis except
-          ``connection.uri`` and ``topics``.
-
+       |
+       | For example, the ``topic.override.foo.collection=bar`` setting instructs 
+       | the sink connector to store data from the ``foo`` topic in the ``bar``
+       | collection.
+       |
+       | You can specify any valid configuration setting in the
+       | ``<propertyName>`` segment on a per-topic basis except
+       | ``connection.uri`` and ``topics``.
+       |
        | **Default**:  ``""``
        | **Accepted Values**: Accepted values specific to the overridden property