apache/airflow

Under which category would you file this issue? Airflow Core Apache Airflow version 3.2.2 What happened and how to reproduce it? When running GCSToGCSOperator with move_object=True, if a task fails after the file has been successfully copied and deleted from the source bucket (e.g., not acknowledge due to network issue), a task retry will attempt to delete the source object again. Because the object was already deleted in the previous run, the hook.delete() call raises a google.api_core.exceptions.NotFound (404) error, causing the task retry to fail completely rather than succeeding idempotently. Logs of error: [2026-08-28 19:07:45] INFO - Object application/interface/myfile.xml in bucket professional-bucket-europe-west1-in rewritten to object ... [2026-08-28 19:07:45] INFO - Blob application/interface/myfile.xml deleted. [2026-08-28 19:07:45] INFO - Executing copy of ... ... [2026-08-28 19:07:55] WARNING - Blob application/interface/myfile.xml in bucket professional-bucket-europe-west

Apache Airflow version 3.0.6 If "Other Airflow 2 version" selected, which one? No response What happened? Running DatabricksRunNowOperator(deferrable=True) (and this likely applies to DatabricksSubmitRunOperator too, since both share the same trigger/hook code) submits the Databricks job successfully and the task defers correctly (worker slot released). But the triggerer then crashes while polling job status, before the run ever reaches a terminal state: ERROR - Trigger failed: Traceback (most recent call last): File ".../airflow/jobs/triggerer_job_runner.py", line 963, in cleanup_finished_triggers result = details["task"].result() File ".../airflow/jobs/triggerer_job_runner.py", line 1072, in run_trigger async for event in trigger.run(): File ".../airflow/providers/databricks/triggers/databricks.py", line 90, in run run_state = await self.hook.a_get_run_state(self.run_id) File ".../airflow/providers/databricks/hooks/databricks.py", line 514, in a_get_run_state

GlueJobOperator._find_job_run_id_by_task_uuid (the task-UUID scan fallback, used when neither task_state_store nor a cached XCom id has a prior run) has a few pre-existing quality issues, carried over verbatim from main, that are worth cleaning up: Unbounded pagination. It's a while True over get_job_runs(MaxResults=50) with no page cap and no age cutoff. The no-match case — the prior attempt died before it ever called StartJobRun, which is the common retry shape — is exactly the one that walks the job's entire run history before giving up. Failure is silently swallowed. The two except Exception blocks around this scan (and the XCom lookup before it) log at warning level and return None, after which the operator submits fresh — straight into ConcurrentRunsExceededException against a run that's still alive. There's no error-level signal telling anyone why. The except Exception is broader than it needs to be. It should be narrowed to except ClientError, so a real bug in the surrou

ReDoS in _mask_cmd() via User-Controlled application_args Summary The apache-airflow-providers-apache-spark package contains a Regular Expression Denial of Service (ReDoS) vulnerability in the SparkSubmitHook._mask_cmd() method. User-controlled values injected through the Airflow REST API DAG trigger conf parameter are incorporated into a shell command string that is processed by a regex with catastrophic backtracking characteristics, enabling a remote DoS attack. Affected Package Package: apache-airflow-providers-apache-spark PyPI: https://pypi.org/project/apache-airflow-providers-apache-spark/ Tested version: 4.10.0 CVSS v3.1: 7.5 Severity: High Vulnerability Details In airflow/providers/apache/spark/hooks/spark_submit.py at lines 508–530, the _mask_cmd() method applies the following regex to ' '.join(connection_cmd): re.sub( r'(\S*?(?:secret|password)\S*?(?:=|\s+)([\'"]?))(?:(?!\2\s).)*', r'\1********', ' '.join(connection_cmd) ) connection_cmd includes self._applica

Background Template fields are rendered after an operator's constructor runs. Any logic applied to a template-field parameter's value inside __init__ — validation, type checks, transformation, string interpolation — therefore operates on the un-rendered Jinja expression, not the real value. This is documented in Creating a custom Operator and contributing-docs/05_pull_requests.rst, and it causes real bugs: #69813 (SSHRemoteJobOperator validated a templated remote_base_dir in __init__, so cleanup validation failed for any custom base) is a recent example. ImportantChecks that only ask whether an argument was passed are the exception and belong in __init__. Read False positives below before fixing an entry. The validate-operators-init prek hook previously only verified that template fields are assigned verbatim at the top level of __init__; it could not see validation calls, conditionals, or transformations (and it did not cover sensors, AwsBaseOperator[...] subclasses, or aws_templ