Why $Hx=x-(\rho u^Tx)u$?

Question

A Householder reflection is a matrix of the form $$H=I-\rho uu^T$$ with $\rho=2/\|u\|^2$. Obviously, $Hx=x-\rho uu^Tx$.

Textbook http://www.mathworks.se/moler/leastsquares.pdf says that $$Hx=x-(\rho u^Tx)u$$

However, I can not see the proof of this statement.

Answer 1

Note that $u^t x \in \mathbb R$ is a scalar, and scalar multipliation $(\lambda,u) \mapsto \lambda u$ is usually written on the left, so $$ Hx = x - \rho u u^tx = x - (\rho u^t x)u. $$

Answer 2

because

$$(uu^T)x = u(u^Tx)$$

$u^Tx$ is not a vector.

so

$u(u^Tx) = (u^Tx)u$

Answer 3

Your question boils down to: why is $(uu^T)x = (u^Tx)u$? There are actually four reasons:

1. Matrix multiplication is associative. Hence $(uu^T)x=u(u^Tx)$.
2. Note that $u^Tx$ is a $1\times1$ matrix whose only entry is the scalar $s:=\langle u,x\rangle=\sum x_iu_i$. So, $u^Tx=[s]$, where the pair of square brackets is used to indicate that $u^Tx$ is a matrix.
3. Now comes a point that is both subtle and obvious: $u[s]=su$. Note that the LHS is a matrix product (an $n\times1$ matrix multiplied by a $1\times1$ matrix) while the RHS is a scalar multiplication (the vector $u$ is scaled by the scalar $s$). This equality is perhaps so obvious that it is taken for granted and isn't discussed in most textbooks.
4. Therefore $(uu^T)x=u(u^Tx)=u[s]=su$. Now, if we have identify $s$ with $[s]$, we can rewrite $su$ as $(u^Tx)u$. Hence $(uu^T)x=(u^Tx)u$.