Lsqnonlin doing too small steps and not moving from initial point during optimization

I am using the lsqnonlin function to calibrate parameters of a ground heat conduction model. I am starting with one-parameter optimization. So far, I am using synthetic data to check the functioning of my model setup. I have a temperature field generated with certain ground thermal parameters combination ("true" parameter values). Then I change the value of one of the parameters ("modified" param. values) and I am trying to retrieve its "true" value by minimizing the residual between the temperature field calculated with true and modified parameter set. I do not add any noise so far to the temperature field simulated with modified target parameter.

The objective function for my parameter (here, the heat capacity of rock component of the ground) looks as on the picture below - it is not smooth. I know that the lsqnonlin searches for local minimum only, however, I believe that the algorithm should be able to overcome this kind of "spikes" in the cost function. In fact, no matter how close to the true parameter value I start the optimization, the lsqnonlin does not move at all from this starting point and the optimization does not progress at all. The objective function just looks this spiky all over the possible parameter values.

I am able to produce a very smooth cost function for the parameter when I adjust the settings of the pdepe solver (the AbsTol and RelTol settings), which I use for solving the heat equation. However, this comes for the cost of extremely long computation times which is not acceptable considering that the forward heat model calculation needs to run many times during the optimization. Therefore I am focusing on adjusting the settings of the lsqnonlin function to overcome the local minima and move to a minimum closer to the true value of calibrated parameter (CRock for the original temperature field = 2e6 [J/m3/K], my starting guesses ranging from 1.4e6 to 1.8e6).

During the lsqnonlin run, I am following the progress of the optimization using the Display option. I can see that the objective function value is not decreasing and the lsqnonlin is keeping to choose smaller and smaller steps, getting stuck even closer to the initial guess ("start" value).

I was trying to adjust lsqnonlin options such as the TolX, TolFun as stopping criteria and DiffMinChange setting, but nothing helps to force the lsqnonlin to make bigger steps to ignore these "spikes". I would be extremely grateful for any suggestions and advices since it is quite some time that I am struggling with this problem.

Sonia

[PATHAN]

Same problem here. "Initial Point is the Local Minimum". No increment or decrement in the values. If you got an idea, I will appreciate if you hint us too. Regards P

Answer 1

[Alan Weiss]

There is some documentation on optimizing problems where the objective function is given by a simulation or integration. While it is very basic, perhaps some of the ideas there could help you.

Also, did you sum the squares in the objective function? lsqnonlin wants the objective not to contain the sum of squares.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Sonia]

Thanks for the tip fro the documentation, using an another solver might be a way to go.

I do not sum the squares in the objective function. As required, I submit a vector as an objective function and let the lsqnonlin to square and sum it.

[PATHAN]

Hi Sonia, n Alan.... Do you solved your problem? I am trying the same. lsqnonlin is saying that "Initial point is the local minimum". I hope you may have gone throw lots of optimization stuffs, if you have any idea, please guide on this matter. Hope for motivating reply, and thanks in advance. #IRespectResearhers :-) PMG

[Sonia]

Hi Pathan

I am not sure I resolved the problem, but I did realize some specific issues related to my particular problem (heat conduction, thermal parameters optimization and temperature field simulation).

1) The parameter I was trying to optimize for (here, the heat capacity of rock) is, in fact, rather insensitive parameter in my calculation. Thus, changing this parameter´s value does not influence result of my forward temperature calculation too much. If lsqnonlin evaluates such cost function, it realizes that the sum of squares with modified parameter is nearly the same as sum of squares with initial parameter guess, hence stopping as "the initial point is local minimum".

I don´t know your optimization problem, but if it is relevant in your case, you could try to do sensitivity analysis of your target parameter.

2) If you find that your target parameter is indeed influencing your forward calculation result, you might need to set the " DiffMinChange " option to certain larger values (e.g. 10% of parameter value or else, according to your sensitivity analysis). The default DiffMinChange value is usually too small if your parameter is less sensitive. In such case, you might also want to set upper and lower bounds (in options) on parameter value, just so it doesn´t rocket into physically implausible values.

3) Finally, it might help to scale your parameters, especially if you´re doing multi-parameter calibration. In my case, the rock heat capacity takes very large values (10^6) while all my other parameters are of magnitude 0.1-10. As far as I could see, you can only specify one value for DiffMinChange , so you need to make sure that your parameters are of the same magnitude and similarly affected by DiffMinChange .

One last thing I realized would apply if you´re working with synthetic optimization dataset (e.g. like I was, in order to prove that my optimization setup works) and pde solvers. In such a case, you want to make sure that calculation of your optimization dataset (the synthetic dataset that you are optimizing on) is more precise then your forward calculations during the optimization. Meaning that your synthetic optimization dataset should be calculated with more strict solver settings (in my case " AbsTol " and " RelTol " settings of pdepe solver) and then you can use looser tolerances for the fwd calculations (which also speeds up the calculation).

Good luck! Hope I was any clear and let know if any of above helped, if you found other solutions or in case you need further details.

Sonia

Answer 2

[David]

Agree with most of the above although (not sure if parameter names have changed) the parameter in the options to modify is called 'FiniteDifferenceStepSize' rather than DiffMinChange . For me setting this to 1e-2 rather than the default of sqrt(eps) (i.e.1e-8) got it to actually do something rather than just not doing anything and saying that the initial point is (a local) optimal.

options=optimoptions('lsqnonlin','display','iter-detailed','FiniteDifferenceStepSize',1e-2);
[x,resnorm,residual,exitflag,output,lambda,jacobian]=lsqnonlin(fun,x0,lb,ub,options);