Merged, formatted

docs/CONTRIBUTING.md

@@ -17,58 +17,73 @@ Here are some tips to get started using the project dependencies and development
 2. Make sure your Python version is 3.10 or 3.11
 
 Fedora 41:
+
 ```
 sudo dnf install python3.11
 ```
+
 2. [Install pip](https://pip.pypa.io/en/stable/installation/)
 
 Fedora 41:
+
 ```
 sudo dnf install pip
 ```
+
 3. [Install pipx](https://pipx.pypa.io/stable/installation/)
 
 Fedora 41:
+
 ```
 sudo dnf install pipx
 ```
+
 4. [Install Poetry](https://python-poetry.org/docs/#):
+
 ```
 pipx install poetry
 ```
+
 Fedora 41:
+
 ```
 sudo dnf install poetry
 ```
 
 5. In the **airbyte-python-cdk project** install [Poe the Poet](https://poethepoet.natn.io/) and unit tests' prerequisites:
+
 ```
 poetry install --all-extras
 ```
 
 6. If your operating system is RHEL or compatible, execute:
+
 ```
 # just for the current session, until restart
 sudo modprobe iptable_nat
 
 # include the nat module to survive restart
 echo iptable_nat | sudo tee -a /etc/modules-load.d/modules
 ```
-See also:
-  * [Dager-Podman Integration](https://blog.playgroundtech.io/introduction-to-dagger-6ab55ee28723)
-  * [CDK Issue 197](https://github.com/airbytehq/airbyte-python-cdk/issues/197)
 
-7. Make sure Docker is installed locally, instead of Podman 
 See also:
-  * [CDK Issue 197](https://github.com/airbytehq/airbyte-python-cdk/issues/197)
+
+- [Dager-Podman Integration](https://blog.playgroundtech.io/introduction-to-dagger-6ab55ee28723)
+- [CDK Issue 197](https://github.com/airbytehq/airbyte-python-cdk/issues/197)
+
+7. Make sure Docker is installed locally, instead of Podman
+   See also:
+
+- [CDK Issue 197](https://github.com/airbytehq/airbyte-python-cdk/issues/197)
 
 8. Edit airbyte/airbyte-integrations/connectors/source-shopify/acceptance-test-config.yml and change:
+
 ```
    connector_image: airbyte/source-shopify:dev    to    connector_image: airbyte/source-shopify:<version>
 ```
-where the **version** comes from airbyte/airbyte-integrations/connectors/source-shopify/metadata.yaml, as 
-the value of the **dockerImageTag** parameter.
 
+where the **version** comes from airbyte/airbyte-integrations/connectors/source-shopify/metadata.yaml, as
+the value of the **dockerImageTag** parameter.
 
 ## Local development
 
@@ -97,52 +112,61 @@ the value of the **dockerImageTag** parameter.
 ### More tools and options
 
 To see all available scripts and options, run:
+
 - `poetry run ruff`
 - `poetry run pytest --help`
 - `poetry run poe`
 
-
 ### Test locally CDK Changes against Connectors (integration tests)
 
 When developing a new feature in the CDK, you may find it necessary:
-* to run a connector that uses that new feature
-* or use an existing connector to validate a new feature
-* or fix it in the CDK.
+
+- to run a connector that uses that new feature
+- or use an existing connector to validate a new feature
+- or fix it in the CDK.
 
 In this project, the [GitHub pipelines](.github/workflows/connector-tests.yml) run such tests against the Shopify source as **integration tests**.
 
 **Assumptions:**
-* The test connector is in the [Airbyte project](https://github.com/airbytehq/airbyte).
-* The [Airbyte project](https://github.com/airbytehq/airbyte) is checked out in `airbyte` directory.
-* The [CDK development](https://github.com/airbytehq/airbyte-python-cdk) project is checked out in the `airbyte-python-cdk` directory - a sibling of the `airbyte` directory.
+
+- The test connector is in the [Airbyte project](https://github.com/airbytehq/airbyte).
+- The [Airbyte project](https://github.com/airbytehq/airbyte) is checked out in `airbyte` directory.
+- The [CDK development](https://github.com/airbytehq/airbyte-python-cdk) project is checked out in the `airbyte-python-cdk` directory - a sibling of the `airbyte` directory.
 
 **Preparation steps**
-* In the `airbyte` project run:
+
+- In the `airbyte` project run:
+
 ```
 cd airbyte/airbyte-integrations/bases/connector-acceptance-test/
 poetry install
 ```
-* Edit the `airbyte/airbyte-integrations/connectors/<test connector>/pyproject.toml` file.
+
+- Edit the `airbyte/airbyte-integrations/connectors/<test connector>/pyproject.toml` file.
   Replace the line with `airbyte_cdk` with the following (see the assumptions above):
+
 ```toml
 airbyte_cdk = { path = "../../../../airbyte-python-cdk", develop = true }
 ```
-* In `airbyte/airbyte-integrations/connectors/<test connector>` reinstall `airbyte_cdk` from your local working directory:
+
+- In `airbyte/airbyte-integrations/connectors/<test connector>` reinstall `airbyte_cdk` from your local working directory:
+
 ```
 cd airbyte/airbyte-integrations/connectors/<test connector>
 poetry install
 ```
-* In `airbyte/airbyte-integrations/connectors/<test connector>/` create the `secrets/config.json` file.
-  For example, use the Shopify connector for the integration test: 
-  * Register in shopify.com as of [Shopify test connector](https://docs.airbyte.com/integrations/sources/shopify):
-  * Use the generated **Admin API Access Token** as the API **password** in the configuration file. 
-  * On the **Settings / Domains** page find the subdomain of *myshopify.com* and use it as the **shop** in the configuration file.
+
+- In `airbyte/airbyte-integrations/connectors/<test connector>/` create the `secrets/config.json` file.
+  For example, use the Shopify connector for the integration test:
+  - Register in shopify.com as of [Shopify test connector](https://docs.airbyte.com/integrations/sources/shopify):
+  - Use the generated **Admin API Access Token** as the API **password** in the configuration file.
+  - On the **Settings / Domains** page find the subdomain of _myshopify.com_ and use it as the **shop** in the configuration file.
     Example:
-    ```
-    domain: nw0ipt-vr.myshopify.com
+    `     domain: nw0ipt-vr.myshopify.com
     shop: nw0ipt-vr
-    ```
-Example contents:
+    `
+    Example contents:
+
 ```
 {
   "shop": "nw0ipt-vr",
@@ -153,13 +177,15 @@ Example contents:
   }
 }
 ```
-* See also:
-  * [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
-  * [Connector Acceptance Tests](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/bases/connector-acceptance-test/README.md)
+
+- See also:
+  - [Acceptance Tests Reference](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
+  - [Connector Acceptance Tests](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/bases/connector-acceptance-test/README.md)
 
 **Steps:**
 
-* Run the connector's tests (see the connector's README.md)
+- Run the connector's tests (see the connector's README.md)
+
 ```
 cd airbyte/airbyte-integrations/connectors/<connector name>
 
@@ -169,7 +195,9 @@ poetry run <connector name> discover --config secrets/config.json
 poetry run <connector name> read --config secrets/config.json --catalog integration_tests/<connector name>.json
 poetry run pytest
 ```
+
 Example:
+
 ```
 cd airbyte/airbyte-integrations/connectors/source-shopify
 
@@ -179,13 +207,17 @@ poetry run source-shopify discover --config secrets/config.json
 poetry run source-shopify read --config secrets/config.json --catalog integration_tests/configured_catalog.json
 poetry run pytest
 ```
-* Run the acceptance tests locally:
+
+- Run the acceptance tests locally:
+
 ```
 cd airbyte/airbyte-integrations/bases/connector-acceptance-test
 
 poetry run pytest -p connector_acceptance_test.plugin --acceptance-test-config=../../connectors/<connector name>
 ```
+
 Example:
+
 ```
 poetry run pytest -p connector_acceptance_test.plugin --acceptance-test-config=../../connectors/source-shopify
 
@@ -196,24 +228,24 @@ poetry run pytest -p connector_acceptance_test.plugin --acceptance-test-config=.
 poetry run pytest -p connector_acceptance_test.plugin --acceptance-test-config=../../connectors/source-shopify --timeout=30
 ```
 
-* When testing is complete, revert your test changes.
+- When testing is complete, revert your test changes.
 
 ### Test CDK Changes against Connectors
 
 **Preparation steps**
 
-* Install the [`airbyte-ci` CLI](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
+- Install the [`airbyte-ci` CLI](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md)
 
 **Steps:**
 
-* Build your connector image with the local CDK using:
+- Build your connector image with the local CDK using:
 
 ```bash
 cd airbyte
 airbyte-ci connectors --use-local-cdk --name=<CONNECTOR> build
 ```
 
-* Use the `test` command with `--use-local-cdk` to run the full set of connector tests, including connector acceptance tests (CAT) and the connector's unit tests:
+- Use the `test` command with `--use-local-cdk` to run the full set of connector tests, including connector acceptance tests (CAT) and the connector's unit tests:
 
 ```bash
 cd airbyte
@@ -222,7 +254,6 @@ airbyte-ci connectors --use-local-cdk --name=<CONNECTOR> test
 
 Note that the local CDK is injected at build time, so if you make changes, you must rerun the build command to see them reflected.
 
-
 ## Working with Poe Tasks
 
 The Airbyte CDK defines common development tasks using [Poe the Poet](https://poethepoet.natn.io/). You can run `poetry run poe list` to see all available tasks. This will work after `poetry install --all-extras` without additional installations.

unit_tests/sources/declarative/extractors/test_dpath_extractor.py

@@ -8,13 +8,6 @@
 import pytest
 import requests
 
-from airbyte_cdk import Decoder
-from airbyte_cdk.sources.declarative.decoders.json_decoder import (
-    IterableDecoder,
-    JsonDecoder,
-    JsonlDecoder,
-)
-from airbyte_cdk.sources.declarative.extractors.dpath_extractor import DpathExtractor
 from airbyte_cdk import Decoder
 from airbyte_cdk.sources.declarative.decoders.json_decoder import (
     IterableDecoder,

unit_tests/sources/declarative/extractors/test_record_extractor.py

@@ -7,10 +7,10 @@
 
 from airbyte_cdk.sources.declarative.extractors.record_extractor import (
     SERVICE_KEY_PREFIX,
+    assert_service_keys_exist,
     exclude_service_keys,
     is_service_key,
     remove_service_keys,
-    assert_service_keys_exist,
 )
 
 

unit_tests/sources/declarative/extractors/test_record_selector.py

@@ -10,12 +10,12 @@
 
 from airbyte_cdk.sources.declarative.decoders.json_decoder import JsonDecoder
 from airbyte_cdk.sources.declarative.extractors.dpath_extractor import (
-     DpathExtractor,
-     RECORD_ROOT_KEY
+    RECORD_ROOT_KEY,
+    DpathExtractor,
 )
 from airbyte_cdk.sources.declarative.extractors.record_extractor import (
-    exclude_service_keys,
     assert_service_keys_exist,
+    exclude_service_keys,
 )
 from airbyte_cdk.sources.declarative.extractors.record_filter import RecordFilter
 from airbyte_cdk.sources.declarative.extractors.record_selector import RecordSelector
@@ -114,7 +114,7 @@
         (
             "test_the original response is available in filters and transformations",
             ["data"],
-            "{{ record['created_at'] == record['"+RECORD_ROOT_KEY+"'].data[1].created_at }}",
+            "{{ record['created_at'] == record['" + RECORD_ROOT_KEY + "'].data[1].created_at }}",
             {
                 "data": [
                     {"id": 1, "created_at": "06-06-21"},