Automatic Differentiation Using Dual Numbers
Forward mode automatic differentiation is accomplished by augmenting the algebra of real numbers and obtaining a new arithmetic. An additional component is added to every number which will represent the derivative of a function at the number, and all arithmetic operators are extended for the augmented algebra. The augmented algebra is the algebra of dual numbers. Computer programs often implement this using the complex number representation.
Replace every number with the number, where is a real number, but is nothing but a symbol with the property . Using only this, we get for the regular arithmetic
and likewise for subtraction and division.
Now, we may calculate polynomials in this augmented arithmetic. If, then
where denotes the derivative of with respect to its first argument, and, called a seed, can be chosen arbitrarily.
The new arithmetic consists of ordered pairs, elements written, with ordinary arithmetics on the first component, and first order differentiation arithmetic on the second component, as described above. Extending the above results on polynomials to analytic functions we obtain a list of the basic arithmetic and some standard functions for the new arithmetic:
and in general for the primitive function ,
where and are the derivatives of with respect to its first and second arguments, respectively.
When a binary basic arithmetic operation is applied to mixed arguments — the pair and the real number — the real number is first lifted to . The derivative of a function at the point is now found by calculating using the above arithmetic, which gives as the result.
Read more about this topic: Automatic Differentiation
Famous quotes containing the words numbers, dual and/or automatic:
“Publishers are notoriously slothful about numbers, unless theyre attached to dollar signsunlike journalists, quarterbacks, and felony criminal defendents who tend to be keenly aware of numbers at all times.”
—Hunter S. Thompson (b. 1939)
“Thee for my recitative,
Thee in the driving storm even as now, the snow, the winter-day
Thee in thy panoply, thy measurd dual throbbing and thy beat
Thy black cylindric body, golden brass and silvery steel,”
—Walt Whitman (18191892)
“The ruin of the human heart is self-interest, which the American merchant calls self-service. We have become a self- service populace, and all our specious comfortsthe automatic elevator, the escalator, the cafeteriaare depriving us of volition and moral and physical energy.”
—Edward Dahlberg (19001977)