Steepest Descent Method Help

I have a code here that generates a symmetric random matrix, as well as the code for the steepest descent method. I now wish to plot the convergence of this optimization method. Could somebody please kindly explain how I can go about doing this? Much appreciated!

n = 40;             % system size
rand('seed',sum(100*clock));  % update the rand seed
A = rand(n);        % generate the random matrix
A = A'*A+5*eye(n);  % generate the symmetric matrix
b = rand(n,1);      % generate the right hand side
x_ext = A\b;        % the exact solution (minimizer)

x = zeros(n,1);     % starting point
err_sd = sqrt((x-x_ext)'*A*(x-x_ext));
for k=1:10
    p = b-A*x;
    alpha = p'*p/(p'*A*p);
    x = x + alpha*p;
    err_sd = [err_sd sqrt((x-x_ext)'*A*(x-x_ext))];
end

Answer 1

[Hari]

Hi Ryan,

I understand that you have implemented the steepest descent method for a symmetric random matrix and now wish to plot the convergence of this optimization method over iterations.

I assume you want to visualize how the error, defined as the Euclidean norm of the difference between the current estimate and the exact solution, decreases over iterations.

In order to plot the convergence of the steepest descent method, you can follow the below steps:

Initialize Your Variables:

Ensure that you have initialized the error vector err_sd to store the error at each iteration. This will be used to plot the convergence.

err_sd = sqrt((x-x_ext)'*A*(x-x_ext)); % Initial error

Iterate and Update the Error:

In your loop, update the solution x using the steepest descent method and append the new error to err_sd.

for k = 1:10
    p = b - A*x;
    alpha = (p'*p) / (p'*A*p);
    x = x + alpha*p;
    err_sd = [err_sd sqrt((x-x_ext)'*A*(x-x_ext))]; % Append new error
end

Plot the Convergence:

Use the “plot” function to visualize the error over iterations.

plot(0:10, err_sd, '-o');
xlabel('Iteration');
ylabel('Error');
title('Convergence of Steepest Descent Method');
grid on;

Example Explanation:

The plot will show how the error decreases as the number of iterations increases, indicating the convergence of the method.

Interpret the Plot:

Analyze the plot to understand the rate of convergence and how quickly the method approaches the exact solution.

Refer to the documentation of “plot” function to know more about the properties supported: https://www.mathworks.com/help/matlab/ref/plot.html

Hope this helps!