Find the maximum value of the function $G(u)=\cfrac{u^TA u}{u^Tu}$ over $R^3$\ {0}

Question

Find the maximum value of the function of

$G(u)$=$\cfrac{u^TAu}{u^Tu}$ over $\Bbb R^3\setminus 0_3$ ($0_3$ in this case is the zero vector in $\Bbb R^3$

we know that $A$=$\begin{bmatrix}3 & 2 &3 \\2&3&2\\2 & 2&3 \end{bmatrix}$

To solve this, first I let $G(u)$=$\begin{bmatrix}x\\y\\z\end{bmatrix}$， so $u^TAu$ = $\begin{bmatrix}x&y&z\end{bmatrix}$$\begin{bmatrix}3 & 2 &3 \\2&3&2\\2 & 2&3 \end{bmatrix}$$\begin{bmatrix}x\\y\\z\end{bmatrix}$

but next I don't know how to do. What is $u^T$ and $u$ in this question?

Answer 1

u and $u^T$ are the vectors $\begin{bmatrix}x\\y\\z \end{bmatrix}$ and $\begin{bmatrix}x&y&z \end{bmatrix}$. The function G(u) is a quadratic form, but are you sure about the restriction?

Answer 2

Let's find a solution to the optimisation problem:

maximise $\Phi(u) = u^TAu $ subject to $u^Tu=1$ for $u \in \mathbb{R}^3$ and some matrix $A$.

For this constrained optimisation problem, one can write the Lagrangian as:

$L(u,\lambda) = u^TAu - \lambda(u^Tu-1)$

For optimality,

$\frac{\partial}{\partial u}L(u,\lambda) = 2Au - 2\lambda u = 0$, implying $Au = \lambda u$.

$\frac{\partial}{\partial \lambda} L(u,\lambda) = 1-u^Tu = 0$, implying $u^Tu=1$.

In other words, one of the solution(s) to $Au = \lambda u$ subject to $u^Tu = 1$, i.e. one of the normalised eigenvectors of $A$ will maximise $\Phi(u)$, and ultimately, $g(u)$.

Answer 3

Consider the slightly more general problem involving two symmetric matrices. $$\max \bigg(\frac{u^TAu}{u^TBu}\bigg)$$ Define scalar variables for the numerator and denominator. $$\eqalign{ \alpha &= u^TAu \implies d\alpha = 2(Au)^Tdu \cr \beta &= u^TBu \implies d\beta = 2(Bu)^Tdu \cr }$$ Write the quadratic form in terms of these variables, then find its differential and gradient. $$\eqalign{ \lambda &= \beta^{-1}\alpha \cr d\lambda &= \beta^{-2}(\beta\,d\alpha-\alpha\,d\beta) &= 2\beta^{-2}(\beta Au-\alpha Bu)^Tdu \cr \frac{\partial\lambda}{\partial u} &= 2\beta^{-2}(\beta Au-\alpha Bu) \cr }$$ Setting this gradient to zero leads to an eigenvalue equation. $$\eqalign{ \beta Au &= \alpha Bu \cr (B^{-1}A)u &= (\beta^{-1}\alpha)u \cr Lu &= \lambda u \cr }$$ So the min/max value of the function $\lambda$ is the min/max eigenvalue of $(B^{-1}A)$.
For your particular problem, $\,\,B=I\,$ and $\,\lambda=G(u)$.


In your problem statement, I see that $A$ is not symmetric. If that's not a typo, then in the above derivation, replace the matrix with its symmetric part: $\,A\rightarrow\big(\frac{A+A^T}{2}\big)$

For the symmetrized matrix, $\,\lambda_{max}=7.3423292$

Answer 4

If $u_0$ is an answer to this problem, then so is $ku_0$ for $k\ne 0$. Therefore this problem can be converted to $$\max x^TAx\\s.t. \\x^Tx=1$$using Lagrange's multiplier's method we obtain:$$2Ax=2\lambda x$$which by substitution leads to $$x^TAx=x^T\lambda x=\lambda x^Tx=\lambda$$therefore $$\max_{x^Tx=1} x^T Ax=\max_{\lambda \text{ is an eigenvalue of }A}\lambda=\lambda_{\max{ }}$$