Why BPTT ≈ Costate?

How the BPTT backward pass recovers the Pontryagin costate.

A technical note behind PG-DPO: the backward pass of backpropagation through time (BPTT) computes the pathwise discrete adjoint of the time-discretized control problem. Under stationarity and martingale-projection consistency, this discrete adjoint converges to the adjoint BSDE — the continuous-time Pontryagin costate. The derivation below makes this correspondence precise.

Why BPTT ≈ Costate — figure 1
Why BPTT ≈ Costate — figure 2
Why BPTT ≈ Costate — figure 3
Why BPTT ≈ Costate — figure 4
Why BPTT ≈ Costate — figure 5
Why BPTT ≈ Costate — figure 6
Why BPTT ≈ Costate — figure 7

← Back to Research