Adjoints by Overloading

When implementing adjoint mode by overloading the overloaded operators and functions build at runtime a separate representation of the program's executiion trace (a version of the directed acyclic computational graph G). A special interpreter is called to linearize G and to propagate adjoints backwards from the outputs to the inputs. The major drawback of this approach is the interpretation step that is hard to optimize for any compiler.

Example

The following computational graph illustrates the computation of the first row of the Jacobian of

x0=x0*sin(x0*x1)
y1=sin(x0)
y0=x0*x1

at point (2.0,3.0). The local partial derivatives are attached to the edges in the linearized computational graph. Adjoints are propagated backwards. Hence, the computational graph needs to be stored or partly recomputed.

-- UweNaumann - 16 Aug 2007