How to Compute the Hessian Matrix of a Scalar-Valued Function

Key Takeaways

• The Hessian matrix is the matrix formed by all the second derivatives of a multivariable function. 

• The Hessian matrix is a symmetric square matrix of order ‘n’ when computed for an n variable function. 

• In optimization problems, the Hessian matrix is computed to obtain critical points such as the maxima/minima of the multivariable function of interest. 

Mathematical modeling is extensively used in engineering and technological systems, as it enables us to describe real-life problems in mathematical terms. Mathematical expressions and functions are solved using various algorithms, and the solution is optimized for better results. Optimization plays a major role in solving engineering problems based on mathematical models. The solution obtained should satisfy the constraints and produce the maximum output and efficiency.

In the process of optimization, most methods compute the Hessian matrix. The Hessian matrix is one way to reach the global optimum of a system expressed as a mathematical function. Let’s explore the Hessian matrix and how to compute it.

What Is the Hessian Matrix?

Multivariable functions are common in mathematical models describing engineering systems. From the second derivative of a multivariable function, it is possible to get an idea about the second-order behavior of the function. Second-order derivatives are important in multivariable functions, as they help determine the critical points in optimization.

Usually, Hessian matrices are calculated to understand the behavior of the functions that are dependent on more than one value. The Hessian matrix is the matrix formed by all the second derivatives of a multivariable function. For a function of n variables, the Hessian matrix is an n x n square matrix. As the order of differentiation does not bring any change in the derivative, the Hessian matrix obeys the condition of symmetry. The Hessian matrix is a symmetric square matrix of order ‘n’ when computed for an n variable function. The generalized Hessian matrix (Hf) is given below.

The Hessian Matrix and Scalar-Valued Functions

A physical space where a function associates a single number to every point forms the scalar field. The scalar-valued function may take more than one input value but always returns a single value. When the scalar-valued function depends on more than one value, it forms a multivariable function. The calculation of the Hessian matrix makes sense only for scalar-valued functions. In the Hessian matrix, each element is a function, and the same is evaluated at some point, say (x₀.y₀,...).

The  aforementioned Hessian matrix generalization can be rewritten as one below for the point (x₀.y₀,...), given which is present in the scalar field as Hf (x₀.y₀,...): 

How to Compute the Hessian Matrix

Consider a differentiable function f: Rⁿ→R. The Hessian matrix for this function can be calculated by following the steps given below.

1. Take the gradient of the function f. Let the gradient of the function be  ▽f : Rⁿ→R

The matrix formed by the first-order partial derivatives is called the Jacobian matrix or gradient matrix. Let’s assume all the partial derivatives are present for the given function.

2. Take the gradient or derivative of the matrix ▽f. The result obtained is a square matrix of order n and it forms the Hessian matrix of f.

3. The Hessian matrix at a given point (x₀.y₀,...) can be calculated by substituting the values in the elements of the Hessian matrix.

Example: Computing the Hessian Matrix 

For example, let’s compute the Hessian matrix for:

Step 1: Compute the first-order partial derivatives.

Step 2: Compute the second-order partial derivatives.

The Hessian matrix of the function is: 

Step 3: Evaluate the Hessian matrix at (x,y)=(1,2)

The Hessian Matrix Is Essential for Optimization

In optimization problems, we compute the Hessian matrix to obtain critical points such as the maxima/minima of the multivariable function of interest. In engineering, the Hessian matrix is essential for image processing, computer vision, and frequency calculation in spectroscopy, among other things. The Hessian matrix is computed in most optimization algorithms.

Cadence CFD tools can assist you in solving optimization problems in highly complex engineering systems. Subscribe to our newsletter for the latest CFD updates or browse Cadence’s suite of CFD software, including Fidelity and Fidelity Pointwise, to learn more about how Cadence has the solution for you.