Difference between SQP and Active Set? I thought active set was a method used within SQP

I believe SQP is a method that iteratively solves QP subproblems.

When solving QP problems, one has to deal with inequality constraints. Active set is a method to iteratively find which constraints are active, so that the problem can be simplified to equality constraints, and maybe fewer constraints as well (inactive constraints might have been removed in the process). Therefore, I thought that Active Set was just a method used in the solving of the subproblems within SQP.

However, MATLAB fmincon can be set to run "active set" and "sqp" as two different methods. SQP is explained within the active set method, and then SQP is again presented as a separate algorithm altogether.

( https://www.mathworks.com/help/optim/ug/constrained-nonlinear-optimization-algorithms.html )

Could somebody clarify the difference? Why is SQP a method in itself, but also a method used within Active Set? Why is it not the other way around?

A high-level explanation would be great.

Answer 1

[Danish Ali]

Hi, Mitsu! You have raised a good point here.

I was wondering about it as well couple of months back. Here is brief explanation from my side,

An Active set algorithm is a part of Sequential Quadratic Programming (SQP) algorithm. Consider it as an upgraded version as "SQP combines two fundamental algorithms for solving non-linear optimization problems: an active set method and Newton’s method"[1].

The reason for using SQP algorithms were introduced due to limitations of active set i.e., inefficiency to solve highly non-linear derivative experssion (comes from Karush–Kuhn–Tucker conditions). To cope of with this issue, SQP uses Newton–Raphson method.

In the fmincon SQP Algorithm section of MATLAB documentation, they have defined SQP from a perspective of how different it is from Active set algorithm.

[1]https://optimization.mccormick.northwestern.edu/index.php/Sequential_quadratic_programming#The_Active_Set_Method_and_its_Limitations