Help for a proof with vectors and matrix

Question

I need help to solve this exercise, I do not want the solution but just some hints.

Given a matrix $A\in \mathbb{R}^{n\times d}$ and collections $\{v_{i}, … ,v_{k}\}$ and $\{w_{i}, … ,w_{k}\}$ orthogonal unit vectors with the same span. Show that $$\sum_{i=1}^k ||Av_{i}||^2=\sum_{i=1}^k ||Aw_{i}||^2$$

My reasoning so far: the matrix $A$ multiplied the vector $v_{i}$ results in an another vector - I'll call it $y_{i} = (y_{i1},..., y_{ik})$ - , the squared norm of that vector is $y_{i1}^2 + ... + y_{ik}^2$. I then tried to substitute v in the result with a linear combination of ${w_{i}, … ,w_{k}}$ but at some point I get stuck.

Answer 1

Hint: There is a nice approach here if we frame the question using matrices instead of vectors. Let $V$ denote the matrix with columns $v_1,\dots,v_k$, and let $W$ denote the matrix with columns $w_1,\dots,w_k$.

Because $\{v_1,\dots,v_k\}$ and $\{w_1,\dots,w_k\}$ are orthonormal sets with the same span, we can deduce that there exists a $k \times k$ orthogonal matrix $Q$ for which $W = VQ$.

Now, note that $$ \sum_{i=1}^k \|Av_i\|^2 = \operatorname{trace}([AV]^T[AV]), \quad \sum_{i=1}^k \|Aw_i\|^2 = \operatorname{trace}([AW]^T[AW]). $$

---

Regarding the problem of showing that there exists a $k \times k$ orthogonal matrix $Q$ for which $W = VQ$: there are a few approaches we can take here. One approach that I like is as follows.

• Because $V,W$ have orthonormal columns, $V^TV = W^TW = I_k$ (the size $k$ identity matrix)
• Because $V,W$ have the same column-spaces, $VV^T$ and $WW^T$ are both an orthogonal projection onto the same subspace. So, $VV^T = WW^T$.

It follows that if we take $Q = V^TW$, then $$ VQ = V(V^TW) = (VV^T)W = (WW^T)W = W(W^TW) = W. $$