how can i validate my optimization algorithm?
12 views (last 30 days)
Show older comments
harsh Brar on 5 May 2022
Commented: Andreas Apostolatos on 7 May 2022
I am using Genetic algorithm, Particle swarm, simulated annealing for comparative study. I want to validate the model. for this purpose let's suppose i have two variable x, y.
their actual values are x = 10, y= 20. and their function returns value 1.33.
in order to validate what i am doing is i am setting lower and upper bound in the range of their respective value. for example for x my lower and upper bound is [5 15] and for y [15 25] and after optimizing these i got 1.55. then i calculate percentage error from estimated and predicted value.
is this is the correct way for validation for model?
Andreas Apostolatos on 5 May 2022
From your description I understand that your cost function f depends on (x,y), namely f = f(x,y) and that f(10, 20) = 1.33 is your initial guess.
If the objective function is non-convex, then it may contain multiple minima/maxima. In such a case constraining your design space might yield only one of these minima/maxima ignoring the rest of the design space. This can help convexify the problem at hand, but it may yield not the most optimal solution in terms of the objective value.
The algorithms you mentioned, namely, the genetic algorithm, the particle swarm, and the simulated annealing are all algorithms of the Global Optimization Toolbox and best suited for problems where the global minimum/maximum is sought (oftentimes one seeks only local minima/maxima where other algorithms are better suited).
My guess would be that if all these algorithms yield the same result, then it is very likely that the solution found is indeed the global optimum. The reason for this argumentation is that these algorithms are inherently different and thus having all of them yield the same result would be a good indication that the converged solution is indeed an optimum.
Since your problem is two dimensional, you can select a set of points (x_i,y_j), compute the value of the objective function at these points f(x_i,y_j) and then plot the objective function as a surface using for instance function surf. You can then plot also the solution from the latter algorithms on this graph and visually verify that the point (x,y) found by these algorithms is a stationary point. You can also compute the gradient of the converged point at that location by means of Finite Differences or so (you would need some points around the converged solution for this and then use function diff for instance) to verify that this is a stationary point.
I hope this information helps.
Alan Weiss on 6 May 2022
Your objective function is smooth. Therefore, you should not be using particleswarm or ga or simulannealbnd, but instead should use a gradient-based solver from Optimization Toolbox™ such as fmincon. See Table for Choosing a Solver. And I think that you made a typo giving the upper and lower bounds the same values, which will keep any algorithm from moving from that point.
The only possible thing to explore is the multiple minima of your objective function, if there are multiple minima. To do so you can use MultiStart, started from a large number of points because you have a lot of dimensions in your problem.
MATLAB mathematical toolbox documentation
Andreas Apostolatos on 7 May 2022
As Alan just mentioned, a gradient-based solver is more suitable for optimization problems whose cost function is smooth, which appears to be the case for you also. You can actually reproduce the solution from particleswarm function using fmincon with the following lines of code,
options_fmincon = optimoptions('fmincon', 'Display', 'iter', 'PlotFcn', 'optimplotx', 'StepTolerance', 1e-12, 'OptimalityTolerance',1e-12);
x0 = mean(lb + ub, 1);
[x_fmincon, fval_fmincon, exitflag_fmincon, output_fmincon] = ...
fmincon(obj, x0, , , , , lb, ub, , options_fmincon)
It is not possible to know in advance whether your cost function has multiple optima, especially for high dimensional design spaces as in your case. Citing again Alan's answer above, the only way to methodologically check for multiple minima would be to use the MultiStart starting from a large number of points, see the following documentation page,
I hope this helps.
Find more on Solver Outputs and Iterative Display in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!