Prefect 3.6.22: New Flow States and Subprocess Monitoring

Here at industry sources, we tracked Prefect's 3.6.22 release and found two material improvements for production workflows. The update introduces two new flow states - Submitting and InfrastructurePending - that emit from BaseWorker. These states fill a visibility gap between when a flow is scheduled and when it actually starts executing on infrastructure. Previously, operators had dead zones in their flow lifecycle monitoring where they couldn't tell if delays were happening at submission, infrastructure initialization, or elsewhere.

The second addition is OS-level resource metric collection for flow run subprocesses. When flows execute as subprocess workers, Prefect now automatically captures CPU, memory, I/O, and network metrics at the operating system level. This moves monitoring beyond application-level logging into actual system resource consumption - critical for debugging performance bottlenecks and capacity planning.

• Submitting state: Emitted when BaseWorker begins infrastructure submission
• InfrastructurePending state: Emitted while waiting for infrastructure to be ready
• Subprocess metrics: CPU, memory, I/O, and network collection per flow run
• Backward compatible - existing deployments won't break

The two new states solve a concrete operator problem: hidden latency. In production orchestration, a flow that's queued for 30 seconds is a critical incident if you don't know why. Is it the queue system? The cloud provider? Network config? The old state machine didn't distinguish. Now you can trace whether a delay is happening at submission (Submitting state delay) or at infrastructure spin-up (InfrastructurePending state delay). This distinction is the difference between debugging your code versus debugging your infrastructure.

Subprocess metric collection addresses the black box problem in worker monitoring. When flows run as subprocesses - common in Prefect Cloud deployments - you were blind to actual system resource usage. You saw flow-level metrics but not what the OS saw. A flow logging 2GB memory usage might actually be consuming 5GB at the system level due to child processes or memory fragmentation. Now you get the OS perspective, which means you can actually right-size your workers and catch resource exhaustion before it crashes deployments.

• Pinpoint infrastructure bottlenecks instead of guessing
• Avoid over-provisioning workers based on incomplete metrics
• Catch resource leaks that application monitoring misses
• Set accurate SLO targets with real infrastructure data

Upgrading to 3.6.22 requires no configuration changes - new states and metrics collect automatically. However, operators should take three concrete actions. First, audit your existing monitoring dashboards and alerting rules. If you have alerts on flow start time, they'll now fire at Submitting instead of Run state. Update thresholds to account for the new intermediate states. Second, add subprocess metric panels to your observability stack. If you're using Prometheus or Datadog, these new metrics should appear in your Prefect telemetry exports. Third, establish baselines. Run 50+ production flows through the new version and record normal ranges for Submitting and InfrastructurePending state durations.

For teams running large-scale deployments with many subprocess workers, the resource metrics unlock capacity planning improvements. Export the subprocess metrics to your cost tracking system. Correlate memory and CPU peaks with deployment size changes. Use this data to set autoscaling policies or worker count targets. This is where the data becomes actionable - not just observability theater, but actual infrastructure optimization.

• Update alerting rules to account for new intermediate states
• Export subprocess metrics to existing monitoring stack
• Establish baseline performance patterns with 50+ flow runs
• Use resource data to tune autoscaling policies

This release reflects Prefect's continued focus on observability as competitive differentiator. The new states and metrics aren't flashy features - they're operational necessity. Competing orchestration platforms (Airflow, Dagster, Temporal) are all pushing similar visibility improvements. Prefect's advantage here is that BaseWorker emits these states natively, meaning they work across all worker types without custom instrumentation. That integration depth matters when you're managing thousands of flows.

The subprocess metrics addition also signals Prefect's commitment to multi-cloud deployments. Subprocess workers are lightweight and don't require Kubernetes - perfect for teams running on traditional VMs or hybrid infrastructure. By improving subprocess observability, Prefect is making the simpler deployment path more enterprise-grade. This is smart product positioning - making sure operators don't feel forced into Kubernetes complexity if they don't need it. The momentum in this space continues to accelerate.

• Observability is becoming table-stakes for orchestration platforms
• Native state emissions win against custom monitoring bolt-ons
• Subprocess support gains feature parity with Kubernetes deployments