Positive definite matrix, least square minimization

Hi, I am trying to solve a constrained least square minimization problem, which will give me a X_{vec}=(15 x 1) matrix. I will be later converting these 15 elements to a symmetric (5 x 5) matrix called X. Is there any way to constriant the elements of the solution X_{vec} of the constrained LSQ minimization problem such that my X will be a positive definite matrix? Thanks a lot in advance!

[reen2015]

I use lsqlin() to solve my constrained least square minimization problem.

Answer 1

[Torsten]

Add the constraints that the five sub-determinants of X have to be positive and use fmincon to solve.

Best wishes

Torsten.

[reen2015]

Hi, Torsten, Thank you for your quick response. Kindly please tell me why you suggest fmincon solver instead of lsqlin?

[Torsten]

lsqlin can not handle nonlinear constraints which result from the conditions on the submatrices.

Best wishes

Torsten.

[reen2015]

my constriants are linear are like, x1>0, x2<0, x3<0, x1+x2+x3>0.

[Torsten]

Your constraints are

X_11 > 0
X_11*X_22-X_12*X_21 > 0
etc.

which are nonlinear in the matrix coefficients.

Best wishes

Torsten.

[reen2015]

oh sorry, I was mentioning the constriants of my original problem, I missed the new constraints for subdeterminants to be positive which are nonlinear.

Thank you so much. But I guess finding sub determinants for 5 x 5 matrix is tough.

Thank you for your time :)

[Torsten]

Not that tough:

    det(X(1:1,1:1)) > 0  
    det(X(1:2,1:2)) > 0
    det(X(1:3,1:3)) > 0
    det(X(1:4,1:4)) > 0
    det(X(1:5,1:5)) > 0

Best wishes

Torsten.

[reen2015]

Thank you so much :)

[reen2015]

Hi,

I did the following

 myfun = @(x) 0.5 * (norm(C*x - d))^2;
        %[x,fval]=fmincon(myfun,x0,AA,bb);%without nonlinear constraints
       nonlcon=@mynon;
               [x,fval]=fmincon(myfun,x0,AA,bb,[],[],[],[],nonlcon);

function [c,ceq]=mynon(x)

c=[-x(1);
    -det([x(1) x(2);x(2) x(6)]); 
  -det([x(1) x(2) x(3);x(2) x(6) x(7);x(3) x(7) x(10)]);
  -det([x(1) x(2) x(3) x(4);x(2) x(6) x(7) x(8);x(3) x(7) x(10) x(11);x(4) x(8) x(11) x(13)]);
  -det([x(1) x(2) x(3) x(4) x(5);x(2) x(6) x(7) x(8) x(9);x(3) x(7) x(10) x(11) x(12);x(4) x(8) x(11) x(13) x(14);x(5) x(9) x(12) x(14) x(15)]);];

ceq=[];

Where AA and bb related to linear inequality constriants. Aeq=[], beq=[] lb=[],ub=[]. Kindly note that matrix X is a 5 x 5 symmetric matrix and constructed out of elements of vector x which is 15 x 1 (no repeating elements).

Optimization using fmincon is not giving any error. However, the matrix I get is not positive definite :(.

Please help! where did I go wrong!

[Torsten]

What are the values for the 5 determinants you get from the final solution ?

Best wishes

Torsten.

[reen2015]

Please see the nonpositive definite matrix X that I got as below

X=[     0.9581   -0.0000   -0.0000    0.0236   -0.3652
   -0.0000    0.1237    0.1707    0.1595    0.2443
   -0.0000    0.1707    0.3126    0.4316    0.2893
    0.0236    0.1595    0.4316    0.0092    0.1186
   -0.3652    0.2443    0.2893    0.1186    0.4588

];

oned=det(X(1:1,1:1))  
twod=det(X(1:2,1:2)) 
    threed=det(X(1:3,1:3)) 
   fourd= det(X(1:4,1:4)) 
    fived=det(X(1:5,1:5)) 

      eig(X)

oned =

    0.9581

twod =

    0.1185

threed =

    0.0091

fourd =

   -0.0071

fived =

    0.0013

ans =

   -0.3122
   -0.0282
    0.1499
    0.8290
    1.2240

[Torsten]

Strengthening the variable TolCon might help.

Best wishes

Torsten.

[reen2015]

Dear Torsten

Thank you so much for your time. MIne is an iterative process, and so I tried pausing in each iteration. here is what I get

Warning: The default trust-region-reflective algorithm does not solve problems with the constraints you have specified. FMINCON will use the active-set algorithm instead. For information on applicable algorithms, see Choosing the Algorithm in the documentation. > In fmincon at 504 In sir7 at 148 Warning: Your current settings will run a different algorithm (interior-point) in a future release. > In fmincon at 509 In sir7 at 148

Solver stopped prematurely.

fmincon stopped because it exceeded the function evaluation limit, options.MaxFunEvals = 1500 (the default value).

Thanks alot for your time.

[reen2015]

I tried changing options using following

 options = optimoptions('fmincon','Algorithm','sqp');
       options=optimoptions(options,'Tolx',1e-10)
       options.MaxFunEvals = 3000;%default=1500
       options.TolCon=1.0000e-4;%default=1.0000e-9

but still the matrix output is not positive definite !!!

[Torsten]

1. If possible, start with a feasible solution.

2. Strengthening the tolerances means: Choose a smaller value than the default (i.e. options.TolCon < default value)).

Best wishes

Torsten.

[reen2015]

Thank you Torsten for your kind suggestions and time :)