Creating Orthogonal, Orthonormal Vectors

Question

I'm trying to understand gram-schmidt orthogonalization process. And it says $p_1,p_2...p_N$ are linearly independent vectors and $q_1,q_2...q_N$ are orthonormal vectors created from them. And also there are some equations like in the below:

$q_1$ is unit vector so it's length is 1 hence $q_1 = p_1/ |p_1|$ and also q1.q1 = 1 because it is unit. But there is an equation that I didn't understand. What means the $$q_2 = a*p_1 + b*p_2 = alpha*q_1 + beta*p_2$$ How was this equality achieved? Is a and b are coefficient/number or angle? What is the geometric explanation for this?

Can anybody explain this equation?

Answer 1

Recall your statement :

And it says $p_{1},p_{2},...,p_{N}$ are linearly independent vectors and $q_{1},q_{2},...,q_{N}$ are orthonormal vectors $\color{blue}{\text{created from them}}$

Indeed, since this is the ultimate goal of the Gram-Schmidt process! To construct a set of orthonormal vectors from your original set of vectors as I highlighted your comment in blue. These two sets should share most of the vector properties.

[Edit] : To make things more clear, I will for the sake of simplicity reduce the set of LI vectors $V$ to just two vectors. Define : $$ V:=\{v_{1},v_{2}\} $$ This set forms a basis in $\mathbb{R}^{2}$. We shall construct the desired set : $$ Q:=\{q_{1},q_{2}\} $$ such that the elements of $Q$ are the orthonormal vectors of $V$ and note that : $$ \operatorname{span}(V)=\operatorname{span}(Q) $$ The GSP begins by defining the first element of $Q$ that is $q_{1}$ : $$ w_{1}:=v_{1}\implies q_{1}:=\frac{w_{1}}{\|w_{1}\|_{2}}=\frac{v_{1}}{\|v_{1}\|_{2}} $$ Next, let $w_{2}:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}$. Geometrically, the projection is more of a "copy" of $v_{2}$ but by forcing it to be aligned in the same direction of $v_{1}$. It turns out that $w_{2}$ is formed by vector addition of $v_{2}$ and $-\operatorname{proj}_{v_{1}}v_{2}$. Therefore, what you get essentially is : \begin{align} w_{2}&:=v_{2}-\operatorname{proj}_{v_{1}}v_{2}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{v_{1}^{\intercal}v_{1}}v_{1}\\ &=v_{2}-\frac{v_{1}^{\intercal}v_{2}}{\|v_{1}\|_{2}^{2}}v_{1}\\ &=v_{2}-(q_{1}^{\intercal}v_{2})q_{1} \end{align} and what remains is normalizing the vector : $$ q_{2}:=\frac{w_{2}}{\|w_{2}\|_{2}}=\frac{v_{2}-(q_{1}^{\intercal}v_{2})q_{1}}{\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}} $$ Now for what you are concerned about are the coefficients. Well note that : $$ v_{1}=\|v_{1}\|_{2}q_{1}=c_{1}q_{1} $$ and \begin{align} v_{2}&=\|v_{2}-(q_{1}^{\intercal}v_{2})q_{1}\|_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1}\\ &=c_{2}q_{2}+(q_{1}^{\intercal}v_{2})q_{1} \end{align} as you can see $q_{1},v_{1}$ and $q_{2},v_{2}$ are all related through $c_{1}$ and $c_{2}$ and this affirms our early statement that the span of $V$ and $Q$ must be the same. In fact, we can write : $$ \mathbf{V}=\begin{bmatrix}v_{1}&v_{2}\end{bmatrix}_{2\times 2}=\begin{bmatrix}q_{1}&q_{2}\end{bmatrix}_{2\times 2}\begin{bmatrix}c_{1}&q_{1}^{\intercal}v_{2}\\ 0&c_{2}\end{bmatrix}_{2\times 2}=\mathbf{QR} $$ where $Q^{\intercal}Q=I$ (i.e., orthonormal matrix) and $R$ is an upper triangular $\color{blue}{\text{invertible}}$ matrix (it is clear why this matrix of coefficients must be invertible).

From this you can generalize this for all $n$ vectors of $V$ in $\mathbb{R}^{n}$.

Answer 2

I will elaborate what the author means. We can first normalize $p_1$ to get $$q_1 = \frac{1}{|p_1|} p_1$$ This $\{ q_1 \}$ is now a 1-element orthonormal set with the same span as $p_1$. It is trivially "orthogonal" and it is normal because $$\langle q_1 , q_1 \rangle = \langle \frac{1}{|p_1|} p_1, \frac{1}{|p_1|} p_1 \rangle = \frac{1}{|p_1|^2} \langle p_1, p_1 \rangle = \frac{|p_1|^2}{|p_1|^2} = 1$$ Now, let's move on to adding $p_2$ into our orthonormal collection. We want $\{ q_1, q_2 \}$ to have the same span as $\{ p_1, p_2 \}$, so if our desired $q_2$ exists, we need it to satisfy $$q_2 = a p_1 + bp_2 = \alpha q_1 + \beta p_2$$ for some $a, b, \alpha, \beta \in \mathbb{R}$ (or $\mathbb{C}$ if you are working with complex vector spaces). In order to ensure orthogonality we need $$0 = \langle q_1, q_2 \rangle = \langle q_1, \alpha q_1 + \beta p_2 \rangle = \alpha \langle q_1, q_1 \rangle + \beta \langle q_1, p_2 \rangle = \alpha + \beta \langle q_1, p_2 \rangle$$ In order to ensure that a possible $q_2$ is normal, we would need $$1 = \langle q_2, q_2 \rangle = \langle \alpha q_1 + \beta p_2, \alpha q_1 + \beta p_2 \rangle = \alpha^2+2 \alpha \beta \langle q_1, p_2 \rangle + \beta^2 |p_2|^2$$ Putting these conditions together, we need $$0 = \alpha + \beta \langle q_1, p_2 \rangle, 1=\alpha^2+2 \alpha \beta \langle q_1, p_2 \rangle + \beta^2 |p_2|^2$$ for the desired $q_2$ to exist. If you solve for $\alpha, \beta$, you will find that they do produce a valid $q_2$ (i.e. $\{q_1, q_2 \}$ is an orthonormal set with the same span as $\{ p_1, p_2 \}$), showing existence.

Edit: I am updating my answer with the new screenshot to follow their method. After seeing $$0 = \alpha + \beta \langle q_1, p_2 \rangle \implies \alpha = -\beta \langle q_1, p_2 \rangle$$ They substitute this into the desired expression for $q_2$: $$q_2 = \alpha q_1 + \beta p_2 = (-\beta \langle q_1, p_2 \rangle) q_1 + \beta p_2 = \beta (p_2 - \langle q_1, p_2 \rangle q_1)$$ They are now saying define vector $r_2$ to be $p_2 - \langle q_1, p_2 \rangle q_1$. Then, by our conditions and definitions, $q_2$ will be orthogonal to $q_1$ exactly when $q_2 = \beta r_2$ where $\beta$ is the $\beta$ we want to define $q_2$ with. Since we also want $|q_2|=1$, we also need $$1 = \langle q_2, q_2 \rangle = \langle \beta r_2, \beta r_2 \rangle = \beta^2 |r_2|^2 \implies \beta = \frac{1}{|r_2|} \implies q_2 = \frac{r_2}{|r_2|}$$ Unraveling our definitions, we have that $$q_2 = \frac{1}{|p_2 - \langle q_1, p_2 \rangle q_1|} (p_2 - \langle q_1, p_2 \rangle q_1)$$ and you can check that this $\{ q_1, q_2 \}$ indeed is an orthonormal set with the same span as $\{ p_1, p_2 \}$.

Answer 3

If $q_1 = \frac {p_1}{\vert p_1 \vert }$ then $aq_1 = a \frac {p_1}{\vert p_1 \vert } = \frac {a}{\vert p_1 \vert }p_1$ so set $\alpha = \frac{a}{\vert p_1 \vert }$ and you're done.