Trouble implementing BFGS
Show older comments
Hello,
I am currently working on a system where I have to find the best signal other a certain time. In this particular problem, the variables to optimize are the amplitudes of the signal for each time. I have implemented the gradient and conjugate gradient method with success but BFGS doesn't seems to work. I use the following code just for the "direction problem": (I need to MAXIMIZE my function that is why the Hessian is negative)
%Define DIRECTION of improvement
HesLength = numel(obj.newGradient);
%Direction for first iteration
if(obj.oldGradient == 0)
obj.newDirection = obj.newGradient;
if(strcmp(type,'BFGS'))
obj.oldHessianInv = -eye(HesLength);
end
else
switch(type)
case 'Gradient'
obj.newDirection = obj.newGradient;
case 'Conjugate'
%Polak-Ribiere method, auto direction reset
Dgrad = obj.newGradient - obj.oldGradient;
beta = (obj.newGradient(:).'*Dgrad(:))/(obj.oldGradient(:).'*obj.oldGradient(:));
beta = max(beta,0);
%beta must be positive, if negative than direction becomes gradient
obj.newDirection = beta*obj.oldDirection + obj.newGradient;
case 'BFGS'
Dgrad = obj.newGradient - obj.oldGradient;
divGrad = obj.deltaPulse(:).'*Dgrad(:);
if(divGrad < 0) %Definite negative condition
QHes = eye(HesLength) - (Dgrad(:)*obj.deltaPulse(:).')/divGrad;
obj.newHessianInv = QHes.'*obj.oldHessianInv*QHes + (obj.deltaPulse(:)*obj.deltaPulse(:).')/divGrad;
obj.newDirection(:) = -obj.newHessianInv*obj.newGradient(:);
obj.oldHessianInv = obj.newHessianInv;
else %If divide by zero use simple gradient
obj.newDirection = obj.newGradient;
obj.oldHessianInv = -eye(HesLength);
end
%direction tends to increase close to convergence.
%When equal inf, convergence is achieved
if(isinf(max(obj.newDirection(:))))
obj.oldDirection = 0;
obj.newDirection = 0;
end
otherwise
ERROR('METHOD NAME IS UNKNOWN');
end
Is there anything wrong for this subpart of the optimization? Also, later in the algorithm the direction is constrained between [-max,+max].
Thanks
Answers (0)
Categories
Find more on Surrogate Optimization in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!