Matlab Code for computing the gradient algorithm

Question

Use the gradient descent algorithm with a fixed step size to find minimizer of the function

$$f\left(x_1, x_2\right) = 100\left(x_2-{x_1}^2\right)^2+\left(1-x_1\right)^2$$

Use the initial condition of $x(0)=(-2, 2)^T$. Choose an appropriate step size $\alpha$ (so that the method works). Terminate the algorithm when the norm of $\nabla f$ is less than $0.0001$.

1. Find (the numerical values of) the minimizer and $\nabla f$ at the minimizer.

2. Find the total number of iterations and values of the minimizer and $\nabla f$ at several (about 10) intermediate steps.

3. Draw a figure to show the contour lines of $f$, the initial guess and the final point, and several intermediate estimates of the minimizer.

Answer 1

This dos not do the plotting in step 3, but provides an implementation of the steepest descent algorithm.

clear;
clc;
close all;

x    = [-2; 2];
grad = f_grad(x);

alpha = 1e-3;
tol   = 1e-4;

n_updates = 0;

while (norm(grad) > tol)
  x         = x - alpha * grad;
  grad      = f_grad(x);
  n_updates = n_updates + 1;
end

fprintf("Minimum is: %f with gradient (%f, %f) \n", f(x), grad(1), grad(2) );
fprintf("Iterations needed for step size %f are: %i \n", alpha, n_updates);

function y = f(x)
  y = 100 * (x(2) - x(1)^2)^2 + (1 - x(1) )^2;
end

function grad_y = f_grad(x)
  grad_y = [-2 * (1 - x(1)) - 400 * (x(1) * x(2) - x(1)^3);
            200 * (x(2) - x(1)^2)];
end