Automatic Differentiation - Automatic Differentiation Using Dual Numbers

Automatic Differentiation Using Dual Numbers

Forward mode automatic differentiation is accomplished by augmenting the algebra of real numbers and obtaining a new arithmetic. An additional component is added to every number which will represent the derivative of a function at the number, and all arithmetic operators are extended for the augmented algebra. The augmented algebra is the algebra of dual numbers. Computer programs often implement this using the complex number representation.

Replace every number with the number, where is a real number, but is nothing but a symbol with the property . Using only this, we get for the regular arithmetic

and likewise for subtraction and division.

Now, we may calculate polynomials in this augmented arithmetic. If, then

where denotes the derivative of with respect to its first argument, and, called a seed, can be chosen arbitrarily.

The new arithmetic consists of ordered pairs, elements written, with ordinary arithmetics on the first component, and first order differentiation arithmetic on the second component, as described above. Extending the above results on polynomials to analytic functions we obtain a list of the basic arithmetic and some standard functions for the new arithmetic:

and in general for the primitive function ,

where and are the derivatives of with respect to its first and second arguments, respectively.

When a binary basic arithmetic operation is applied to mixed arguments — the pair and the real number — the real number is first lifted to . The derivative of a function at the point is now found by calculating using the above arithmetic, which gives as the result.

Read more about this topic:  Automatic Differentiation

Famous quotes containing the words numbers, dual and/or automatic:

    Publishers are notoriously slothful about numbers, unless they’re attached to dollar signs—unlike journalists, quarterbacks, and felony criminal defendents who tend to be keenly aware of numbers at all times.
    Hunter S. Thompson (b. 1939)

    Thee for my recitative,
    Thee in the driving storm even as now, the snow, the winter-day
    declining,
    Thee in thy panoply, thy measur’d dual throbbing and thy beat
    convulsive,
    Thy black cylindric body, golden brass and silvery steel,
    Walt Whitman (1819–1892)

    The ruin of the human heart is self-interest, which the American merchant calls self-service. We have become a self- service populace, and all our specious comforts—the automatic elevator, the escalator, the cafeteria—are depriving us of volition and moral and physical energy.
    Edward Dahlberg (1900–1977)