Assume $A\in \mathbb{S}^n$ is a positive semi-definite matrix, and denote its eigenvalues by $\lambda_1 > \lambda_2 \ge ... \ge \lambda_n \ge 0$. Let $w\in \mathbb{R}^n$ be a unit vector such that $$w^TAw=\lambda$$ I would like to upper-bound and lower-bound the quadratic form of $w$ with respect to $A^k$ $$w^TA^kw$$ in terms of $\lambda, \lambda_1, ..., \lambda_n$. More specifically, I would like to obtain a bound for the cases of $k=2,3$.
I've managed to find the following bound $$\lambda_n^{k-1} \cdot \lambda \le w^TA^kw \le \lambda_1^{k-1} \cdot \lambda$$ Proof
Denote the spectral decomposition of $A$ by $A = \sum_{i=1}^n \lambda_i u_i u_i^T$. It is easy to show that the spectral decomposition of $A^k$ is $A = \sum_{i=1}^n \lambda_i^k u_i u_i^T$. And thus $$w^TA^kw = w^T \left( \sum_{i=1}^n \lambda_i^k u_i u_i^T \right) w$$ $$= \sum_{i=1}^n \lambda_i^k w^T u_i u_i^T w$$ $$\le \sum_{i=1}^n \lambda_1^{k-1} \lambda_i w^T u_i u_i^T w$$ $$= \lambda_1^{k-1} \sum_{i=1}^n \lambda_i w^T u_i u_i^T w$$ $$= \lambda_1^{k-1} \cdot \lambda$$ And similarly for the second inequality.
Is there a tighter bound to be obtained? Or one that is expressed solely with $\lambda$? I'm interested specifically in the cases where $k=2,3$.
A positive semidefinite matrix is a Hermitian matrix all of whose eigenvalues are nonnegative.
The finite-dimensional spectral theorem says that any Hermitian matrix can be diagonalized by a unitary matrix, and that the resulting diagonal matrix has only real entries. This implies that all eigenvalues of a Hermitian matrix A with dimension n are real, and that A has n linearly independent eigenvectors. Moreover, a Hermitian matrix has orthogonal eigenvectors for distinct eigenvalues. Even if there are degenerate eigenvalues, it is always possible to find an orthogonal basis of $C^{n}$ consisting of n eigenvectors of A.
So Diagonalizing the matrix by the Unitary matrix U:
$$w^{T}A^{k}w = w^{T}UD^{k}U^{-1}w = w^{T}UD_{k}U^{-1}w$$
Where $D_{k} = D^{k}$ is another diagonal matrix.
If none of the eigenvalues are degenerate you can push the $w$ through $U$ and consider that
$$w^{T}A^{k}w = w^{T}UD_{k}U^{-1}w = v^{T}D_{k}v$$
Where $v = U^{-1}w$. This works since $U$ is unitary and so the adjoint transpose of U is $U^{-1}$.
You can get (possibly in some cases) stronger bounds $v^{T}D_{k}v$ based on what v is here, which should be clear.