how to find the vector that makes the maximum change

Question

For example, $f(x,y)=x^2+y^2$, we know that the maximum change occurs in the direction of the gradient $\nabla f=<2x,2y>=<2,-2>$, but how can I find the vector that makes the maximum change easily? I know it has to be in the tangent plane at the point, but even I work out the tangent plane I still couldn't find an easy way to work out the vector?

Answer 1

It seems that what you’re looking for is basically the projection of the positive $z$-axis onto the tangent plane. We know that the direction of greatest change (in the $x$-$y$ plane) at $(x_0,y_0)$ is given by the gradient $\nabla f$, so we want a positive $z$ such that the vector $\mathbf v=(f_x(x_0,y_0),f_y(x_0,y_0),z)$ is parallel to the tangent plane at the point $P_0=(x_0,y_0,f(x_0,y_0))$. Since $\nabla F(P_0)$, where $F(x,y,z)=f(x,y)-z$ is normal to the tangent plane at $P_0$, the vector $\mathbf v$ that we seek will be orthogonal to $\nabla F$. So, we have $$\nabla F\cdot\mathbf v=(f_x,f_y,-1)\cdot(f_x,f_y,z)= f_x^2+f_y^2-z=0$$ and thus $$z=f_x^2+f_y^2.$$ In your example, $\nabla f=(2x,2y)$, so $z=(2\cdot1)^2+(2\cdot-1)^2=8$.

This of course fails if $\nabla f$ vanishes, but in that case there is no direction of greatest change, anyway.

Answer 2

Your function is actually $$z = x^2 + y^2$$ which can be written as $$F(x, y, z) = x^2 + y^2 - z = 0$$ Its gradient is $\nabla F(x,y,z) = (\partial F/\partial x, \partial F/\partial y, \partial F/\partial z)$, $$\vec{n}(x, y, z) = \nabla F(x,y,z) = (2 x, 2 y, -1)$$ (Note that some write $\nabla F = (\partial F/\partial x, \partial F/\partial y, \partial F/\partial z) = ( F_x, F_y, F_z )$; the subscripts then indicate partial derivatives, and not components of $F$. I personally dislike that notation.)

Normalizing the gradient vector $\vec{n}(x,y,z)$ to unit length gives you a better idea of its direction: $$\hat{n}(x, y, z) = \frac{\vec{n}(x,y,z)}{\lvert\vec{n}(x,y,z)\rvert} = \left( \frac{x}{\sqrt{x^2 + y^2 + \frac{1}{4}}}, \; \frac{y}{\sqrt{x^2 + y^2 + \frac{1}{4}}}, \; -\frac{1}{\sqrt{4 x^2 + 4 y^2 + 1}} \right)$$

Now, to the issues in your problem statement (or rather, assumptions about the solution.)

The surface $F(x,y,z)=0$ is constant; there is no change between different points in the surface (so the fastest change vector is definitely not in the tangent plane). You only see change when you go away or towards the surface. This direction, $\vec{n}(x,y,z)$, is the surface normal.

If you draw the normal vector from point $(x, y, z)$ towards $(x, y, z) + \vec{n}(x, y, z)$, you'll see it always points downwards and away from the $z$ axis (the line from $(x, y, z)$ to $(x, y, z) - \vec{n}(x, y, z)$ always passes through the $z$ axis). It is not the vector you have drawn in your figure; it is perpendicular to it.

$F(x,y,z) = 0$ defines an implicit surface. Because the normal vector $\vec{n}(x, y, z)$ is nonzero everywhere on the surface (because the $z$ component is always -1), the surface is regular. This in turn means that the tangent plane is well defined for every point on the surface. The equation for the plane tangent to the surface at $(x_0, y_0, z_0)$ is $$\vec{n}(x_0, y_0, z_0) \cdot ( x - x_0, y - y_0, z - z_0) = 0$$ i.e. $$2 x_0 (x - x_0) + 2 y_0 ( y - y_0) - (z - z_0) = 0$$ which is equal to $$(2 x_0) x + (2 y_0) y + (-1) z + (z_0 - 2 y_0^2 - 2 x_0^2) = 0$$ I wrote the last form, so you could easily see that it describes a plane perpendicular to the normal vector $(2 x_0, 2 y_0, -1)$ at point $(x_0, y_0, z_0)$. (The scalar part, the last part in parenthesis, determines the signed distance from origin (scaled by the length of the normal vector) along the normal vector to the plane.)

If you want to parametrize the tangent plane -- say, your underlying question is actually about something completely different to the direction of most change --, there are ways to do that, too, but I think that should be a separate question. (The easiest suggestion, using say $v$ for the direction away from the $z$ axis, and $u$ perpendicular to it, is ill-defined at origin -- since the point is on the $z$ axis, every direction is away, much like every direction is "North" on the South pole.)