Update documentation about System Tests (apache#37062)

The documentation was a little out-dated.

contributing-docs/testing/system_tests.rst

@@ -20,16 +20,40 @@ Airflow System Tests
 
 System tests need to communicate with external services/systems that are available
 if you have appropriate credentials configured for your tests.
-The system tests derive from the ``tests.test_utils.system_test_class.SystemTests`` class. They should also
-be marked with ``@pytest.marker.system(SYSTEM)`` where ``system`` designates the system
-to be tested (for example, ``google.cloud``). These tests are skipped by default.
 
-You can execute the system tests by providing the ``--system SYSTEM`` flag to ``pytest``. You can
-specify several --system flags if you want to execute tests for several systems.
+.. contents:: :local:
+
+Purpose of System Tests
+-----------------------
+
+Airflow system tests are pretty special because they serve three purposes:
+
+* they are runnable tests that communicate with external services
+* they are example_dags that users can just copy&paste and use as starting points for their own DAGs
+* the excerpts from these system tests are used to generate documentation
+
+Old System Tests
+----------------
+
+The system tests derive from the ``tests.test_utils.system_test_class.SystemTests`` class.
+
+Old versions of System tests should also be marked with ``@pytest.marker.system(SYSTEM)`` where ``system``
+designates the system to be tested (for example, ``google.cloud``). These tests are skipped by default.
+
+For older version of the tests, you can execute the system tests by providing the
+``--system SYSTEM`` flag to ``pytest``. You can specify several --system flags if you want to execute
+tests for several systems.
 
 The system tests execute a specified example DAG file that runs the DAG end-to-end.
 
-.. contents:: :local:
+New System Tests
+----------------
+
+The new system tests follows
+[AIP-47 New Design of System Tests](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-47+New+design+of+Airflow+System+Tests)
+and those system tests do not require separate ``pytest`` flag to be executed, they also don't need a separate
+class to run - all the code is kept in the system test class that is also an executable DAG, it is discoverable
+by ``pytest`` and it is executable as Python script.
 
 Environment for System Tests
 ----------------------------
@@ -74,13 +98,8 @@ run Google Cloud system tests.
 
   RANDOM_POSTFIX=$(cat "${RANDOM_FILE}")
 
-
-To execute system tests, specify the ``--system SYSTEM``
-flag where ``SYSTEM`` is a system to run the system tests for. It can be repeated.
-
-
 Forwarding Authentication from the Host
-----------------------------------------------------
+---------------------------------------
 
 For system tests, you can also forward authentication from the host to your Breeze container. You can specify
 the ``--forward-credentials`` flag when starting Breeze. Then, it will also forward the most commonly used
@@ -94,75 +113,11 @@ Currently forwarded credentials are:
   * credentials stored in ``${HOME}/.docker`` for ``docker`` client
   * credentials stored in ``${HOME}/.snowsql`` for ``snowsql`` - SnowSQL (Snowflake CLI client)
 
-Adding a New System Test
---------------------------
-
-We are working on automating system tests execution (AIP-4) but for now, system tests are skipped when
-tests are run in our CI system. But to enable the test automation, we encourage you to add system
-tests whenever an operator/hook/sensor is added/modified in a given system.
-
-* To add your own system tests, derive them from the
-  ``tests.test_utils.system_tests_class.SystemTest`` class and mark with the
-  ``@pytest.mark.system(SYSTEM_NAME)`` marker. The system name should follow the path defined in
-  the ``providers`` package (for example, the system tests from ``tests.providers.google.cloud``
-  package should be marked with ``@pytest.mark.system("google.cloud")``.
-
-* If your system tests need some credential files to be available for an
-  authentication with external systems, make sure to keep these credentials in the
-  ``files/airflow-breeze-config/keys`` directory. Mark your tests with
-  ``@pytest.mark.credential_file(<FILE>)`` so that they are skipped if such a credential file is not there.
-  The tests should read the right credentials and authenticate them on their own. The credentials are read
-  in Breeze from the ``/files`` directory. The local "files" folder is mounted to the "/files" folder in Breeze.
-
-* If your system tests are long-running ones (i.e., require more than 20-30 minutes
-  to complete), mark them with the ```@pytest.markers.long_running`` marker.
-  Such tests are skipped by default unless you specify the ``--long-running`` flag to pytest.
-
-* The system test itself (python class) does not have any logic. Such a test runs
-  the DAG specified by its ID. This DAG should contain the actual DAG logic
-  to execute. Make sure to define the DAG in ``providers/<SYSTEM_NAME>/example_dags``. These example DAGs
-  are also used to take some snippets of code out of them when documentation is generated. So, having these
-  DAGs runnable is a great way to make sure the documentation is describing a working example. Inside
-  your test class/test method, simply use ``self.run_dag(<DAG_ID>,<DAG_FOLDER>)`` to run the DAG. Then,
-  the system class will take care about running the DAG. Note that the DAG_FOLDER should be
-  a subdirectory of the ``tests.test_utils.AIRFLOW_MAIN_FOLDER`` + ``providers/<SYSTEM_NAME>/example_dags``.
-
-
-A simple example of a system test is available in:
-
-``tests/providers/google/cloud/operators/test_compute_system.py``.
-
-It runs two DAGs defined in ``airflow.providers.google.cloud.example_dags.example_compute.py``.
-
-
-The typical system test session
--------------------------------
-
-Here is the typical session that you need to do to run system tests:
-
-1. Enter breeze
-
-.. code-block:: bash
-
-   breeze down
-   breeze --python 3.8 --db-reset --forward-credentials
-
-This will:
-
-* stop the whole environment (i.e. recreates metadata database from the scratch)
-* run Breeze with:
-
-  * python 3.8 version
-  * resetting the Airflow database
-  * forward your local credentials to Breeze
-
-3. Run the tests:
-
-.. code-block:: bash
+Running the System Tests
+------------------------
 
-   pytest -o faulthandler_timeout=2400 \
-      --system=google tests/providers/google/cloud/operators/test_compute_system.py
+Running the tests and developing tests in `Test documentation <../../tests/system/README.md>`__
 
------
+------
 
 For other kinds of tests look at `Testing document <../09_testing.rst>`__