Question on applying the constraint $x^Tx=1$ in optimization using eigenvalues of quadratic form $x^TAx$

748 Views Asked by At

I am reading a linear algebra chapter on constrained optimization.

Let A be a symmetric matrix and $x^TAx$ the quadratic form of a quadratic function.
Subject to the constraint $x^Tx=1$, the Max is the largest eigenvalue, $\lambda_{max}$, of A and is attained when x is a unit eigenvector $u_i$ corresponding to $\lambda_{max}$.

I follow the book contents and find this method neat as many functions in engineering and economics are quadratic.
But the book is very light on how to apply the constraint $x^Tx=1$ in practice.
For a constraint having only quadratic terms, I think we can transform it to $x^Tx=1$ using change of variable like this (right?).
$$\begin{align} Constraint(x,y) &= 5x^2 + 7y^2 =28 \\ & = \frac{5x^2}{5\cdot7} + \frac{7y^2}{5\cdot7}=\frac{28}{5\cdot7} \\ &= (\frac{x}{\sqrt{7}})^2 + (\frac{y}{\sqrt{5}})^2 = \frac{28}{35}\\ &= \frac{35}{28}\cdot((\frac{x}{\sqrt{7}})^2 + (\frac{y}{\sqrt{5}})^2) = 1\\ &= (\sqrt{\frac{35}{196}}\cdot x)^2 + (\sqrt{\frac{35}{140}}\cdot y)^2 = 1\\ &= x_1^2+x_2^2 = 1 \qquad\text{for } x_1=\sqrt{\frac{35}{196}}\cdot x, x_2=\sqrt{\frac{35}{140}}\cdot y\\ &= x^Tx =1\\ \end{align}$$

My questions are:

  1. For any single constraint having only quadratic terms, is it always possible like above to use change of variable to make it $x^Tx=1$?
  2. What if the constraint has cross-product terms? How can you make it $x^Tx=1$?
  3. In practice, an objective function is often subject to multiple constraints. Does this linear algebra method still work or not? If yes, how?
  4. Any suggested books/materials where I can learn more, including the above general situations?
1

There are 1 best solutions below

6
On

Economist here

  1. Indeed, you can do it
  2. If you have $x'Ax$ and A is symmetric then use the spectral theorem (it also holds for 1.)
  3. I'm not sure, by my intuition is that it does not work.
  4. This method is very important in economics. Some examples are: factor models, principal components, and Linear-Quadratic approximation of models. In fact, the latter is a very popular way of solving DSGE models