Introduction to Algorithmic Differentiation

Due to the ongoing pandemic it remains unclear to what extend lectures / tutorials can be held in a non-virtual environment. A decision for the whole semester will be made by Oct. 12, 2021 at 2:30pm (first lecture to be held on Zoom). In any case, videos of all lectures and other relevant material are available online (see below).

Contents

  1. Welcome [slides]
  2. Derivative code by overloading
    1. univariate scalar differentiable programs
      1. motivation. essential calculus. finite differences [video_1] [video_2] [slides] [code] (week 2)
      2. first derivative code [video] [slides] [code] (week 3)
      3. second and higher derivative code [video] [slides] [code] (week 4)
    2. multivariate vector-valued differentiable programs
      1. motivation. essential calculus. finite differences [video_1] [video_2] [slides] [code] (week 5) 
      2. first-order tangent code [video_1] [video_2] [slides] [code] (week 6)
      3. second- and higher-order tangent code [video] [slides] [code] (week 7)
      4. first-order adjoint [video_1] [video_2] [slides] [code] (week 8)
      5. second- and higher-order adjoint [video] [slides] [code] (week 9)
  3. Derivative code by hand
    1. tangent code [video] [slides] [code] (week 10)
    2. adjoint straight-line code  [video] [slides] [code] (week 11)
    3. adjoint intra- and interprocedural code [video] [slides] [code] (week 12)
      1. case study [video] [slides] [code]
  4. Derivative code by automatic source transformation
    1. overview and lexical analysis [video_1]  [video_2] [slides] [code] (week 13)
    2. syntax analysis [video] [slides] [code] (week 14)
    3. attribute grammars and syntax-directed translation [video] [slides] [code] (week 15)

Disclaimer: Numerical results obtained during some finite difference experiments (1.2.1. and 1.2.2.) turned out to be erroneous due to hardware problems (now sorted). Correct results should be produced on your own computers. Comments about deviations of finite differences from tangent and adjoint results made in the video to be taken with a "grain of salt".