Suppose I have a function like $f(x,y) = e^x e^y x^2 y^2$, and I want to know in which direction the derivative will grow fastest at a stationary point.
$(0,0)$ is a stationary point of the example formula and the direction in which the gradient grows fastest is $(1,1)$ (or $(\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}})$ when we normalize the direction vector).
Is there a method for finding the direction in which the gradient grows fastest analytically for any function? How would I go about doing such a thing?
Should I look at the third order derivatives?
You have a function $f(x,y)$ and you know that $\nabla f(x_0,y_0)=0$, that is, this is a stationary point, but the methodology is general. The second derivative matrix is $\nabla^2 f(x,y)$, and applying it to a vector will give the rate of change of the gradient in that direction.
Because of the symmetry of partial derivatives, the matrix $\nabla^2 f(x,y)$ is symmetric, and as so, diagonalizable. That means that you can get the direction of fastest changing as the eigenvector connected to the eigenvalue of greatest magnitude.