Skip to content

DAG processor treats any zip-format file (.jar, .pptx, .docx, .xlsx, etc.) as a potential DAG bundle, not just .zip #71125

Description

@seanmuth

Description

find_dag_file_paths (airflow-core/src/airflow/utils/file.py) decides whether to attempt DAG discovery on a file using:

if path.is_file() and (path.suffix == ".py" or zipfile.is_zipfile(path)):
    if might_contain_dag(file_path, safe_mode):
        file_paths.append(file_path)

zipfile.is_zipfile() is a content sniff (checks for the PK zip magic bytes/central directory), not an extension check. There is no accompanying path.suffix == ".zip" condition. Since the ZIP format underlies many common file types beyond .zip itself — .jar, .pptx, .docx, .xlsx, .apk, .epub, .odt, .whl, etc. — any of these dropped into a DAGs folder (e.g. a build artifact, a supporting doc, a packaged dependency) will pass this check and get opened and scanned via might_contain_dag, purely because it happens to share the underlying zip container format with Airflow's own zipped-DAG-bundle feature.

This was reported previously in #45718 with a .pptx file, but that issue was closed as invalid because the specific symptom described there (garbled metric names) turned out to be an unrelated stat-sanitization bug, not this zip-detection design question. The zip-detection behavior itself was never actually addressed.

Impact

At minimum this is wasted work (opening and scanning irrelevant files on every DAG processor cycle). Depending on might_contain_dag's heuristic and the archive's contents, it can also produce confusing log noise, and in the originally reported case, appears able to feed corrupted/unexpected data further into DAG processing.

What you think should happen instead

Gate the zip-bundle branch on the file extension in addition to (or instead of) the content sniff, e.g.:

if path.is_file() and (path.suffix == ".py" or (path.suffix == ".zip" and zipfile.is_zipfile(path))):

This preserves the documented .zip DAG-bundle behavior while no longer opening arbitrary non-.zip files that merely share the same underlying container format. .airflowignore is a viable per-deployment workaround today (matching by extension), but it means every deployment that happens to keep e.g. .jar files anywhere under its DAGs folder has to know to add this rule proactively rather than it being a non-issue by default.

How to reproduce

  1. Place any non-.zip PK-zip-format file (a .jar, .pptx, .docx, etc.) anywhere under the DAGs folder.
  2. Wait for the DAG processor to walk the directory.
  3. Observe that the file is opened and passed through might_contain_dag (visible via DAG processor debug logs), the same as a .zip DAG bundle would be, despite not being one.

Are you willing to submit a PR?

  • Yes

Code of Conduct

  • I agree to follow this project's Code of Conduct

Drafted-by: Claude Code (Sonnet 5); reviewed by @seanmuth before posting

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions