feat: Add OPA support

[siegfriedweber]

Description

part of #446

Add OPA support

• Extend the Airflow clusterConfig with OPA authorization
• Implement an integration test for OPA
• Document the OPA integration

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

Preview Give feedback

1. Changes are OpenShift compatible
2. CRD changes approved (stackabletech/decisions#40)
3. CRD documentation for all fields, following the style guide.
4. Helm chart can be installed and deployed operator works
5. Integration tests passed (for non trivial changes)
6. Changes need to be "offline" compatible

Reviewer

Preview Give feedback

1. Code contains useful comments
2. Code contains useful logging statements
3. (Integration-)Test cases added
4. Documentation added or updated. Follows the style guide.
5. Changelog updated
6. Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

Preview Give feedback

1. Feature Tracker has been updated
2. Proper release label has been added
3. Roadmap has been updated

[razvan]

This PR it's self looks good as is and the tests work with Airflow 2.10.4.

Unfortunately there is an incompatibility with Airflow 2.9.x images:

[2025-02-10T13:30:30.952+0000] {cli_parser.py:78} WARNING - Authentication manager is not configured and webserver will not be able to start.                                                                          
[2025-02-10T13:30:30.953+0000] {cli_parser.py:81} ERROR - The object could not be loaded. Please check "auth_manager" key in "core" section. Current value: "opa_auth_manager.opa_fab_auth_manager.OpaFabAuthManager". 
Traceback (most recent call last):                                                                                                                                                                                     
  File "/stackable/app/lib64/python3.9/site-packages/airflow/configuration.py", line 1214, in getimport                                                                                                                
    return import_string(full_qualified_path)                                                                                                                                                                          
  File "/stackable/app/lib64/python3.9/site-packages/airflow/utils/module_loading.py", line 39, in import_string                                                                                                       
    module = import_module(module_path)                                                                                                                                                                                
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module                                                                                                                                        
    return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                                                        
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import                                                                                                                                                      
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load                                                                                                                                                   
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked                                                                                                                                           
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked                                                                                                                                                    
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module                                                                                                                                              
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed                                                                                                                                         
  File "/stackable/app/lib64/python3.9/site-packages/opa_auth_manager/opa_fab_auth_manager.py", line 21, in <module>                                                                                                   
    from typing import override                                                                                                                                                                                        
ImportError: cannot import name 'override' from 'typing' (/usr/lib64/python3.9/typing.py)                                                                                                                              

I will wait with the approval until the 2.9 images are updated.

[razvan]

I consider this a win:

--- FAIL: kuttl (5087.26s)
    --- FAIL: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-insecure-tls_openshift-false_executor-celery (366.39s)
        --- PASS: kuttl/harness/opa_airflow-2.10.2_opa-latest-0.67.1_openshift-false (224.79s)
        --- FAIL: kuttl/harness/logging_airflow-2.10.2_openshift-false_executor-kubernetes (581.15s)
        --- PASS: kuttl/harness/logging_airflow-2.10.2_openshift-false_executor-celery (312.79s)
        --- PASS: kuttl/harness/orphaned-resources_airflow-latest-2.10.2_openshift-false (212.39s)
        --- PASS: kuttl/harness/overrides_airflow-latest-2.10.2_openshift-false (246.64s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-2.10.2_openshift-false_executor-kubernetes (259.02s)
        --- PASS: kuttl/harness/mount-dags-gitsync_airflow-latest-2.10.2_openshift-false_executor-celery (272.32s)
        --- PASS: kuttl/harness/smoke_airflow-2.10.2_openshift-false_executor-kubernetes (228.87s)
        --- PASS: kuttl/harness/smoke_airflow-2.10.2_openshift-false_executor-celery (250.84s)
        --- PASS: kuttl/harness/cluster-operation_airflow-latest-2.10.2_openshift-false (270.83s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-no-tls_openshift-false_executor-kubernetes (192.61s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-server-verification-tls_openshift-false_executor-celery (225.76s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-no-tls_openshift-false_executor-celery (238.66s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-insecure-tls_openshift-false_executor-kubernetes (200.72s)
        --- PASS: kuttl/harness/ldap_airflow-latest-2.10.2_ldap-authentication-server-verification-tls_openshift-false_executor-kubernetes (201.16s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-2.10.2_openshift-false_executor-kubernetes (153.32s)
        --- PASS: kuttl/harness/mount-dags-configmap_airflow-latest-2.10.2_openshift-false_executor-celery (200.66s)
        --- PASS: kuttl/harness/oidc_airflow-2.10.2_openshift-false (247.67s)
        --- PASS: kuttl/harness/resources_airflow-latest-2.10.2_openshift-false (199.85s)
FAIL
ERROR:root:kuttl failed

[razvan]

lgtm!