Direction of gradient unit vector at r

Question

Consider the scalar field in two dimensions f (x,y) = x2 + 4y2. This field has a global minimum at r = (0, 0). Write a computer program in which, starting from the initial point r(0) = (4, 1.5), you try to reach the minimum of the field by taking small steps ∆t along the negative direction of the local gradient, which is defined by n = −∇f/|∇f|.

I'm having a hard time understanding what is being asked, am I supposed to start at n(4,1.5) and iterate until I reach n(0,0)? Which doesn't really make sense

Answer 1

Start off at vector $r_0=(4,1.5)$. The direction of decrease is in direction of $\nabla f(r_0)$. So, $\Delta t(\nabla f(r_0))$ would be a small change in direction of minimum. So, $r_1=r_0-\Delta t(\nabla f(r_0))$ would be a position vector a little closer to the minimum than $r_0$ was. By defining successive $r_n=r_{n-1}-\Delta t(\nabla f(r_{n-1}))$, you can get successively closer to the minimum.

Answer 2

$ \def\n{\nabla}\def\o{{\tt1}}\def\p{\partial} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} \def\c#1{\color{red}{#1}} \def\fracLR#1#2{\LR{\frac{#1}{#2}}} $Start by writing the function as a matrix-vector equation and calculating its gradient $$\eqalign{ A &= \m{1&0\\0&4}, \qquad &x = \m{x\\y} \\ f(x) &= x^TAx \qiq &g(x)=\n\!f = 2Ax \\ }$$ Then apply a gradient descent iteration like the following.

Initialization $$\eqalign{ x_0 &= \m{4\\1.5} \qquad g_0 = g(x_0)= \m{8\\12} \quad \\ }$$ First step $$\eqalign{ x_1 &= x_0 - 10^{-3}g_0 \quad\qquad\qquad\qquad\qquad \\ g_1 &= g(x_1) \\ }$$ Subsequent steps $$\eqalign{ x_{k+1} &= x_{k} - \left[ \frac{(x_k-x_{k-1})^T(g_k-g_{k-1})}{(g_k-g_{k-1})^T(g_k-g_{k-1})} \right]\c{g_k} \\ g_{k+1} &= g(x_{k+1}) \\ {\rm if}\,\big(g_{k+1}&\approx 0\big)\quad {\rm then\;exit} \\ }$$ The expression in brackets that multiplies the gradient $(\c{g_k})$ is the steplength which characterizes the Barzilai-Borwein method.

Alternatively, you could use fixed steplengths or calculate an optimal steplength via line-searches, but this simple expression is surprisingly effective.