Solving linear/nonlinear systems of equations with the constraint that variables have to be the same value

For a given set of linear/nonlinear systems of equations, I'm looking for a solver that enables the specification of constraints such that two variables of the solution have to be the same. fmincon at first seemed to be in the right direction, but based on my further understanding of the possible constraints it does not seem to accept such constraints.

Any help appreciated!

Answer 1

[John D'Errico]

Are you sure that fmincon not accept that kind of constraint?

What is the purpose of the equality constraints that you can supply? Suppose, for example, that you have 5 variables, and specified

Aeq = [ 1 -1 0 0 0];
beq = 0;

What would that do? (Assuming you passed them in as the equality constraint inputs?) How would it impact variables 1 and 2?

[holistic]

This is so obvious when I see it now, thank you for the hint John :).

[Torsten]

Why do you formulate this as a constraint ?

Why don't you just reduce the number of variables by 1 by inserting x(1) for x(2) ?

[John D'Errico]

(I knowTorsten knows this, but for those who would read the comment and not appreciate the difference in approach....)

You always can reduce the problem by eliminating variables, this is true. At the same time, then you need to reconstruct the set of variables at the end.

For example, suppose you wanted to fix variable 1 to have the value 3 in an optimization. You could artifically replace every instance of x(1) with the number 3 in your code. But that forces you to re-write your code, your objective function. And at the end, you will need to do something like

Xsol = [3,Xsol];

to reconstruct the complete set of variables.

The alternative is to use a linear equality constraint to enforce the value of x(1). (Please avoid the idea of setting a pair of lower and upper bound constraints to the same value. While it would seem this should work, in fact, it can create nasty numerical problems.) Or, here, the goal might be to enforce that x(1)-x(2)=0. Use of simple equality constraints allows the user to enforce a desired result, while not needing to change their base code, or to renumber their variables, then inserting the eliminated variable back into the solution.

The cost of employing such a linear equality constraint is that it leaves fmincon with a slightly more complex problem to solve. In fact, that is not a large cost in the case of linear equality constraints, because you would write an optimization code like fmincon to explicitly eliminate one )or more) of your variables when a linear equality constraint is involved. So, internally, the first thing that fmincon does is to reduce the problem space. It solves the problem in the reduced subspace, then at the very end, it will recover the original variables. (Most likely, this is done internally using a column pivoted QR deomposition on the array Aeq. That handles the completely general case where you may have multiple linear equality constraints. And because of the column pivoting, it is stable and as well posed numerically as we can reasonably make it.)

The cost is relatively small in the end, at a gain of simplicity in how you use the code. But if you truly needed the utmost speed of throughput, you want to do the reduction yourself. That saves fmincon from needing to call something like a QR decomposition on your equality constraint array. The cost of doing the work yourself is seen in terms of programmer time, usually far more expensive than CPU time.

If you were using a tool that did not allow linear equality constraints (fminunc perhaps), then you would often not have a good alternative to reducing the variable search space yourself. But as long as fmincon allows it, you want to at least consider using the linear equality constraints.

One final point, is that you could in theory have used the nonlinear equality constraints in fmincon also. The flaw with that approach is fmincon does not know that your provided nonlinear constraint is truly a linear one. As such, fmincon is then forced to leave the problem in the original space, then at each iteration, numerically differentiating your equality constraint. This is of course much less efficient than one call to QR at the very beginning of the code.

[holistic]

Thank you again! I just started getting into equation solving with matlab, so that was pretty informative for me ;).