Optimization using fmincon with respect to 2 variables

Hello, I'm trying to implement this optimization problem by using fmincon in Matlab . I'm struggling to adapt my constrained conditions in the attached figure to parameters A,b,Aeq,beq, lb,ub,nonIcon in fmincon(fun,x0,A,b,Aeq,beq,lb,ub,nonlcon). Like you can also notice,my function depends on 2 variables HatAlpha and HatC. Please note that HatAlpha is a (m+1 x 1)vector and HatC is a (m+1 x m+1) matrix. This is why I'm also wondering if I have to create a new function variable that contains both of them for fmincon. Thank you for any tipp

Answer 1

[Alan Weiss]

I think that you would find it much easier to use the problem-based approach rather than fmincon directly. The reason is that with fmincon you have to put all of the control variables into one vector, typically called x. This x would contain both HatAlpha and HatC. With the problem-based approach, you are free to use the variables that are most natural for your problem.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Khalil Messaoudi]

Hi Alan,

I tried using the proble-based approach but I'm facing some issues. I defined the variables this way :

Hatalpha = optimvar ('Hatalpha',70);
HatC = optimvar ('HatC',70,70);
prob.Objective = transp(muP- Ksi * Hatalpha) * 1\(Kss + Ksi * HatC * transp(Ksi)) ;

muP is 100*1 vector

Ksi is 100*70 matrix

Kss is 100*100 matrix

I get an error when computing prob.Objective :

'' Error using optim.internal.problemdef.ElementwiseOperator/checkIsValid

Argument dimensions 0-by-0 and 100-by-100 must agree.

Error in optim.internal.problemdef.Ldivide/checkIsValid

Error in optim.internal.problemdef.Ldivide.getLdivideOperator

Error in \ ''

The matrices sizes are fine for multiplication and division, I don't understand why I get this error message. Can you help please ?

[Alan Weiss]

I'm sorry, but I do not understand what you are doing. What is transp? Do you mean it to be the ' operator? And what do you mean by 1\...? Why put in a scalar value here? That looks wrong to me.

Usually, an objective function is a scalar. Do you expect your expression to be a scalar?

Finally, the error indicates that there is a 0-by-0 argument in your expression somewhere. Can you find it?

Good luck,

[Khalil Messaoudi]

Hi Alan,

Yes transp means the ' operator and 1\... means the inverse operator. I changed transp with ' and 1\.... with (...)^(-1) and it looks to compile now. thanks !

However, it looks like the problem-based approach does not support the determinant operation 'det()' on optimization variables even though they have a matrix form. For example :

det(Kss + Ksi * HatC * (Ksi)')^(-1) 

does not compile and I get this error message :

Check for incorrect argument data type or missing argument in call to function 'det'

Is there a way to overcome this issue ?

Thanks

[Torsten]

(muP- Ksi * Hatalpha)' is 1x100

(Kss + Ksi * HatC * (Ksi)') is 100x100

So (muP- Ksi * Hatalpha)' \ (Kss + Ksi * HatC * (Ksi)') is not defined.

If (muP- Ksi * Hatalpha)' was 100x1, then (muP- Ksi * Hatalpha)' \ (Kss + Ksi * HatC * (Ksi)') was defined (although it wouldn't make much sense, I guess).

[Khalil Messaoudi]

Hi Torsten,

that issue was resolved thanks ! Do you have a solution regarding the determinant operation ?

Thanks

[Torsten]

As you said:

The problem-based approach does not support the determinant operation 'det()' on optimization variables.

[Alan Weiss]

Usually there are better alternatives than using det. For example, to solve linear equations you should use \ instead of det.

However, if you need to use det for some reason, you can always use fcn2optimexpr. As explained many times in the doc for problem-based optimization, "If you have a nonlinear function that is not composed of polynomials, rational expressions, and elementary functions such as exp, then convert the function to an optimization expression by using fcn2optimexpr. See Convert Nonlinear Function to Optimization Expression and Supported Operations for Optimization Variables and Expressions."

Alan Weiss

MATLAB mathematical toolbox documentation

[Khalil Messaoudi]

Thanks for the tip.

When trying to solve the optimization problem, I get the following error message

Solver stopped prematurely.

fmincon stopped because it exceeded the function evaluation limit,
options.MaxFunctionEvaluations = 3.000000e+03.

I then enabled iterative display using optimoptions and I get this message after running the code :

Solving problem using fmincon.
                                            First-order      Norm of
 Iter F-count            f(x)  Feasibility   optimality         step
    0    6435    1.450000e+01    1.958e+11    1.086e+06

Can you please explain what these values actually mean and guess what is probably wrong with my problem and where to look for issues ? Thank you

[Alan Weiss]

You can find details of the iterative display in Iterative Display. For your case, the line you show means the following:

• At the initial iteration (iteration 0), fmincon takes 6435 function evaluations, presumably because that is about how many variables you have and it is estimating a gradient by finite difference steps.
• The initial point has an objective function value of 14.5.
• The feasibility at the initial point is about 2e11, which means that the initial point is infeasible (this is a very large value, so the initial point is nowhere close to feasible).
• The estimated first-order optimality measure at the initial poiint is about 1e6, which again is nowhere close to 0, so the initial point is not only infeasible but is not close to a solution.

With this many variables you should probably increase the function evaluation limit to at least 1e5.

options = optimoptions("fmincon","MaxFunctionEvaluations",1e5);

Be sure to pass the options to solve.

[sol,fval] = solve(prob,x0,"Options",options)

It is also possible that your problem is not formulated correctly. Only you know whether these initial values are reasonable or not. But I would be suspicious. In particular, is the feasibility at the initial point expected to be of order 1e11?

Alan Weiss

MATLAB mathematical toolbox documentation

[Khalil Messaoudi]

Hi Alan,

Thanks for your explanation.

Actually, the initial point does not satisfy the constaints, so a very high order feasibility is no surprise.

If you look at the original problem setting :

my minimization problem aims to update HatAlpha and HatC and also reduce their rank through constaints 1 and 2. The first constraint for example makes sure that the last element of updated HatAlpha is 0. So the original HatAlpha before update,which has not a last entry with value 0, does not satisfy for example the first constraint, and is used as my initial point

Is that a problem when using fmincon ? Should the initial point also satisfy the constaints ?

Thank you

[Alan Weiss]

No, fmincon attempts to satisfy the constraints, that is one of its main features. The reason I commented is I rarely see such large constraint violations in a problem, so I wondered if something was wrong in your problem formulation. If you believe that your formulation is correct and that there are indeed feasible points, then there may be nothing wrong.

Alan Weiss

MATLAB mathematical toolbox documentation

[Khalil Messaoudi]

Hi Alan,

Although you told me that it is not necessary for initial point to satisfy the constraints, I got the following error message :

Solving problem using fmincon.
Error using optim.problemdef.OptimizationProblem/solve
Nonlinear constraint function is undefined at initial point. Fmincon cannot continue.

I had to define this function below because the problem-based approach does not support the operator 'det'

fun = @(HatC) det(HatC+Q) ;
c5 = fcn2optimexpr(fun ,HatC)  ;
cons5 = c5 >= 0 ;
prob.Constraints.cons5 = cons5 ;

Is there something wrong with the formulation of the constraint function ? Thanks in advance

[Alan Weiss]

Please read the error message carefully: the constraint function is UNDEFINED at the initial point. The initial point does not have to be feasible, but the constraint must be defined properly.

I suggest using the debugger.

Alan Weiss

MATLAB mathematical toolbox documentation

[Khalil Messaoudi]

Hello,

I run fmincon and I get the message : 'Local minimum found that satisfies the constraints'. But in reality, the given solution actually does not satisfy at least one constraint. For example, one of the constraints makes sure that the last entry of my optimal variable should be 0. But in the given solution by fmincon, I find the entry to be different from 0 (-3,...). How is this possible ?

Thanks for helping

[Alan Weiss]

Constraints are satisfied to within a relative value of the constraint tolerance. This means that the final infeasibility is divided by the initial infeasibility, and that is compared to the constraint tolerance. If your initial infeasibility is 1e11 and the final infeasibility is 1e3, then the relative infeasibility is about 1e-8, which is smaller than the default constraint tolerance of 1e-6.

You can run fmincon again starting from the solution point, I mean something like

[x2.fval2] = fmincon(fun,x1,...)

where x1 represents the first solution.

Alan Weiss

MATLAB mathematical toolbox documentation

[Torsten]

@Khalil Messaoudi

For example, one of the constraints makes sure that the last entry of my optimal variable should be 0. But in the given solution by fmincon, I find the entry to be different from 0 (-3,...). How is this possible ?

Maybe your implementation of the constraint is wrong. Did you set the constraint in Aeq and beq ?

[Khalil Messaoudi]

@Torsten

I'm using the problem-based optimization workflow.

So I set my constraint using prob.Constraints :

cons1 = Hatalpha(d) == 0

and then call it :

prob.Constraints.cons1 = cons1

I guess the formulation is right ?

[Torsten]

I always use the solver-based formulation. So I can't tell.

[Khalil Messaoudi]

@Alan Weiss Is there a way to make fmincon only stop when all the constraits are satisfied ? Would decreasing the default constraint tolerance always make sense ?

[Walter Roberson]

@Khalil Messaoudi

What would you want the code to do in the case where the constraints could not be satisfied?

1. If the lb, ub are inconsisten, or the A, b are inconsistent, or the Aeq, beq are inconsistent ?
2. If it has used lb, ub, A, b, Aeq, beq to break down the search space into all polytope subspaces, and it has explored the function value at boundaries and in the centroid of each of the subspaces, and finds no evidence that the nonlinear constraints can be satisfied?

Suppose I define a nonlinear constraint function

c = @(x) 1/2-(x(1)==3432523.353208).*(x(2)==90071992547409.92)

That function has a well-defined global minima: it is -1/2 at (3432523.353208, 90071992547409.92) and it is 1/2 everywhere else. Keeping in mind that fmincon() effectively cannot examine the source code to determine those constants, if you were to tell fmincon not to stop until all the constraints were satisfied, how long would you want fmincon to search until it happened to try the exact combination of that pair of floating point values?

[Alan Weiss]

I stand by my previous comments. It is clear from the exit message that, as far as fmincon is concerned, the constraints are satisfied at the end. If you disagree, it is because you have a different notion of constraint satisfaction.

I told you earlier that your initial constraint function value was extremely large. That has come back to cause you issues now. I suggest again that you rerun the solver from the final point to get your constraint infeasibility to a better value. You culd also scale your constraint infeasibility by, say, taking a nonlinear transformation such as

consval = 1e3*tanh(consval/1e3);

before you return the value.

Alan Weiss

MATLAB mathematical toolbox documentation