Introduction to Algorithmic Differentiation (AD I)

Contents

  1. Welcome [video] [slides]
  2. Derivative code by overloading
    1. univariate scalar differentiable programs
      1. motivation. essential calculus. finite differences [video_1] [video_2] [slides] [code] (week 1)
      2. first derivative code [video] [slides[code] (week 2)
      3. second and higher derivative code [video] [slides] [code] (week 3)
    2. multivariate vector-valued differentiable programs
      1. motivation. essential calculus. finite differences [video_1] [video_2] [slides] [code] (week 4
      2. first-order tangent code [video_1] [video_2] [slides] [code] (week 5)
      3. second- and higher-order tangent code [video] [slides] [code] (week 6)
      4. first-order adjoint [video_1] [video_2] [slides] [code] (week 7)
      5. second- and higher-order tangent [video] [slides] [code] (week 8)
  3. Derivative code by hand
    1. tangent code; adjoint straight-line code  [video] [slides] [code] (week 9)
    2. adjoint code (intra-/interprocedural control-flow reversal)  [video] [slides] [code] (week 10)
    3. Hello AD World  [video] [slides] [code] (week 11)
  4. Derivative code by automatic source transformation
    1. lexical and syntax analysis  [video] [slides] [code] (week 12)
    2. attribute grammars and syntax-directed translation [video] [slides] [code] (week 13)
    3. syntax-directed derivative code [video] [slides] [code] (week 14)

Disclaimer: Numerical results obtained during some finite difference experiments (1.2.1. and 1.2.2.) turned out to be erroneous due to hardware problems (now sorted). Correct results should be produced on your own computers. Comments about deviations of finite differences from tangent and adjoint results made in the video to be taken with a "grain of salt".