OpenTelemetry support now stable in Airflow 3.1.8

OpenTelemetry support was introduced as experimental in Airflow 2.7.0. After two major versions and multiple refinement cycles, the Airflow project has promoted it to stable status in 3.1.8. This means the API is locked, backward compatibility is guaranteed across minor versions, and the feature has passed production testing.

Stable status doesn't mean feature-complete — it means the foundation is solid enough for production workloads. The fixes and improvements since 2.7.0 addressed integration points with the Airflow execution model, collector compatibility, and trace payload handling.

• Graduated from experimental → stable without breaking public APIs
• Multiple production deployments validated the implementation
• Fixes addressed DAG serialization, task context propagation, and span lifecycle issues

This is a bet on distributed tracing as first-class instrumentation. Instead of relying only on logs and metrics, teams can now trace requests through task execution, operator invocations, and sensor polling. OTel is the industry standard — support for it means Airflow DAG telemetry integrates cleanly with your APM stack (Datadog, New Relic, Jaeger, etc.).

For builders, stability means you can commit to OTel instrumentation without rebuilding in the next Airflow release. This unlocks better debugging of complex DAGs, dependency tracking across systems, and performance analysis at the span level. If you're running multi-tenant or high-concurrency workflows, this is operationally significant.

• Distributed tracing now a supported observability layer alongside logs and metrics
• Drop-in integration with any OTel-compatible APM backend
• Trace context propagation enables cross-system visibility (task → external API → database)

Stable status comes with expectations: the API won't break, the feature will be maintained across releases, and performance regressions trigger rollback reviews. However, stable ≠ feature-rich. OpenTelemetry support in Airflow covers core instrumentation — task execution, operator spans, sensor polling — but edge cases around custom operators or dynamic DAG generation may require custom span logic.

Builders should expect standard OTel semantics: span naming follows OpenTelemetry conventions, attributes map to Airflow domain objects (task_id, dag_id, run_id), and exporters work like any other OTel integration. No Airflow-specific configuration overhead beyond pointing an exporter endpoint.

• Core DAG and task-level spans are automatic; custom operators need custom instrumentation
• Sampling, batching, and export configuration delegated to OTel SDK — no new Airflow knobs
• Trace schema stable; you can safely write alerting rules on span attributes

If you're already using experimental OTel in Airflow 2.x, upgrading to 3.1.8 is low-risk. The stable version maintains backward compatibility with experimental configurations. Your existing OTEL_EXPORTER settings, environment variables, and instrumentation code remain valid.

For new adopters: the effort is minimal. Enable OTel via environment variables or Airflow config, point it at your collector/APM backend, and spans start flowing. The main work is integrating the traces into your observability workflows — dashboards, alerting, debugging flows. That's where the operator payoff sits.

• Experimental configs work unchanged on 3.1.8; no forced refactoring
• New deployments: set OTEL_SDK_DISABLED=false and an exporter endpoint — that's the minimum
• Biggest effort: building observability workflows (dashboards, SLOs, debugging runbooks)