Science Fair Project Encyclopedia

Science Fair Project Encyclopedia Contents Page

Method of steepest descent

For the optimization method called "steepest descent" see gradient descent.

In mathematics, the steepest descent method or saddle-point approximation is a method used to approximate integrals of the form

$\int_a^b\! e^{M f(x)}\, dx\,$

where f(x) is some twice-differentiable function, M is a large number, and the integral endpoints a and b could possibly be infinite.

Contents

1 The idea of the method

2 General theory

3 Example: Stirling's approximation

4 See also

The idea of the method

Let x₀ be a global maximum of f(x), which, for simplicity, we will assume to be unique. Then, the value f(x₀) will be larger than other values f(x). If we multiply this function by a large number M, the gap between Mf(x₀) and Mf(x) will only increase, and then it will grow "exponentially" for the function

e M f (x) .

As such, significant contributions to the integral of this function will come only from points x in a neighborhood of x₀, and which can be estimated.

General theory

To be able to actually state and prove the method, we need several more assumptions. We will assume that x₀ is not an endpoint of the interval of integration, that the values f(x) cannot be very close to f(x₀) unless x is close to x₀, and that $f''(x 0) < 0$ .

We can expand f(x) around x₀ by Taylor's theorem,

$f(x) = f(x_0) + f'(x_0)(x-x_0) + \frac{1}{2} f''(x_0)(x-x_0)^2 + O\left((x-x_0)^3\right).$

Since x₀ is a global maximum and since it is not an endpoint, it is a stationary point, and therefore f'(x₀) = 0. The function f(x) may be approximated to quadratic order as

$f(x) \approx f(x_0) - \frac{1}{2} |f''(x_0)| (x-x_0)^2$

for x close to x₀. The assumptions we put ensure the accuracy of the approximation

$\int_a^b\! e^{M f(x)}\, dx\approx e^{M f(x_0)}\int e^{-M|f''(x_0)| (x-x_0)^2/2}dx$

where the integral is taken in a neighborhood of x₀. This latter integral is a Gaussian integral if the limits of integration go from −∞ to +∞ (which can be assumed so because the exponential decays very fast away from x₀), and thus it can be calculated. We find

$\int_a^b\! e^{M f(x)}\, dx\approx \sqrt{\frac{2\pi}{M|f''(x_0)|}}e^{M f(x_0)} \mbox { as } M\to\infty.\,$

In extensions of this method, complex analysis is used to find a contour of steepest descent for an equivalent integral, expressed as a path integral.