Exist a linear combination for full rank matrix?

Question

I met a problem this morning.

Suppose that there exists a full rank matrix $X = \begin{bmatrix} a & b & c \\ b & c & b\\ c & d & a \end{bmatrix} \in \mathbb{R}^{3\times3}$. From the definition of linear independence we know that if the matrix is not a full rank matrix, a relationship would exist: $k_1X_1 + k_2X_2 = X_3$, where $X_1$, $X_2$ and $X_3$ are corresponding columns of $X$, and $k_1$, $k_2 \in \mathbb{R}\backslash0$.

If $k_1$ and $k_2$ exist, then: $$\begin{bmatrix} X_1 & X_2\end{bmatrix} \begin{bmatrix} k_1 \\ k_2\end{bmatrix} = \begin{bmatrix} X_3 \end{bmatrix}$$ $$ \begin{bmatrix} k_1 \\ k_2\end{bmatrix} = pinv (\begin{bmatrix} X_1 & X_2\end{bmatrix}) \begin{bmatrix} X_3 \end{bmatrix}$$ $$ \begin{bmatrix} k_1 \\ k_2\end{bmatrix} = (\begin{bmatrix} X_1 \\X_2\end{bmatrix} \begin{bmatrix} X_1 & X_2\end{bmatrix})^{-1}\begin{bmatrix} X_1\\ X_2\end{bmatrix} \begin{bmatrix} X_3 \end{bmatrix}$$

And I try some simulations in MATLAB, the $k_1$ and $k_2$ exist. But the first equation is not satisfied.

So I am very confused. I must be wrong. But where? Could anyone tell me?

Answer 1

It seems you have solved a least square problem finding the projection of $X_3$ onto $span(X_1,X_2)$.

Refer to the related Is the pseudoinverse matrix the solution to the least squares problem?

Answer 2

Your characterization of linear dependence is incorrect. The columns of $X$ are linearly dependent iff there is a nontrivial linear combination $k_1X_1+k_2X_2+k_3X_3$ that vanishes, but this does not let you conclude that $k_1X_1+k_2X_2=X_3$. Consider, for example, $$X=\begin{bmatrix}1&0&0\\0&0&0\\0&0&1\end{bmatrix}.$$ The columns are linearly dependent (every set that contains the zero vector is), but the third column is obviously not a scalar multiple of the first.

If the columns are linearly dependent, then you know that at least one of them is a linear combination of the other two, but you can’t a priori decide which one that is.