5 views (last 30 days)

What is exactly the reason that a Hessian should be positive definite?

Furthermore, if there are polyhedron input and state constraints, the constraints describe a space ''the polyhedron'' where the algorithm should search for a solution.

If you're Hessian is then ill-conditioned and converges therefore slowly, the ''interior-point'' algorithm cannot find a solution in this polyhedron and the algorithm jumps around in the solution space. Why does it then exactly jump around?

John D'Errico
on 23 Mar 2020

Edited: John D'Errico
on 23 Mar 2020

(This is not really a MATLAB question, but you are asking a question about the behavior of MTLAB solvers. Therefore close enough.)

There is no reason that a hessian MUST be positive definite, except at the solution. In fact, in general, the local Hessian may not be pos def, IF you computed it directly. For example, suppose you started the solver near a local max? Then the hessian there would in fact be neg definiite LOCALLY. But the approximation to the Hessian would still be positive definite.

So at a local minimizer of a function, the Hessian must be pos def. At other locations, the true hessian need not be so. However, if you are using a method that updates estimates of the Hessian, then the updates will ensure that the sequences of Hessian approximations are in general pos def. (I seem to recall that.) You can prove that for a line search method (again, as I recall from long ago) that as long as the search direction is a descent direction for methods like BFGS, etc., will always maintain pos def-ness of the Hessian.

In the case where the Hessian estimates are nearly singular during the iterations, then the solver can jump around because the system of equations it solves to choose a new search step are not well posed. So you get garbage,

Sign in to answer this question.

Opportunities for recent engineering grads.

Apply Today
## 0 Comments

Sign in to comment.