[Bug] load_yaml_dags does not call clean_dags

[wearpants]

DAG Factory version

0.19.0, 0.20.0

airflow version

2.9

Python version

3.11

Deployment

Other Docker-based deployment

Deployment details

No response

What happened?

DAGs loaded from yaml files via load_yaml_dags are not removed from airflow when the yaml file is deleted; this behavior is different than what's recommended in README (clean_dags(); generate_dags()`).

Relevant log output

How to reproduce

Create a directory of yaml files and load it, then delete one. It won't disappear from airflow.

Anything else :)?

Not entirely sure I understand what clean_dags is doing, and how to apply it to the multiple-file load_yaml_dags case

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Contact Details

No response

[cmarteepants]

I have a question about this one! We also noticed that clean_dags is not invoked, but in practice noticed that when we removed configs for a specific dag, the dag was deleted - likely because the removal is handled by Airflow natively in your version. It's on our list to look into more deeply as we think the function is there for legacy reasons. Did you experience anything different? Meaning, if you removed configs for a specific DAG while using load_yaml_dags, did the dag go away in Airflow?

We're going to create some official docs, and I had planned to switching to the directory method at that time, but if you confirm that the dag does go away, happy to do it now in the readme to avoid confusion.