I have the following question in an exercise of linear fitting.
Q.
We have $n$ linearly independent known vectors $\textbf{a}_i$, $i = 1, . . . , n$, a known vector $\textbf{b}$, and an unknown vector $\textbf{x}$. All vectors are of dimension $m$. Under what conditions is there a solution to the following equation?
$$\sum_{i=1}^{n} \textbf{a}_i\textbf{a}_i^T \textbf{x}= \textbf{b}$$
Express in terms of $m$ and $n$.
My attempt:
Let $\textbf{A} = [\textbf{a}_1, ..., \textbf{a}_n]$, $m × n$ matrix.
So, $ \sum_{i=1}^{n} \textbf{a}_i\textbf{a}_i^T \textbf{x} =\textbf{A}\textbf{A}^T \textbf{x} = \textbf{b}$.
Since $\textbf{a}_i$ are linearly independent vectors which form the columns of $\textbf{A}$, column rank of $\textbf{A}$ is $n$.
For equation $\textbf{A}\textbf{A}^T \textbf{x} = \textbf{b}$, we have the following 3 cases considering the $m × m$ matrix $\textbf{A}\textbf{A}^T$:
Case 1: $m = n$
$rank(\textbf{A}) = n \implies rank(\textbf{A}\textbf{A}^T) = n = m$
So, $\textbf{A}\textbf{A}^T$ is a full rank matrix, hence, invertible. So the equation has a unique solution.
Case 2: $m > n$
$rank(\textbf{A}) = n \implies rank(\textbf{A}\textbf{A}^T) = n < m$
So, $\textbf{A}\textbf{A}^T$ is a rank deficient matrix. For a solution to exist, vector b must lie in the column space of matrix $\textbf{A}\textbf{A}^T$.
(1) Can we express this in terms of $m$ and $n$?
Case 3: $m < n$
$rank(\textbf{A}) = n \implies rank(\textbf{A}\textbf{A}^T) = n > m$
This case of a matrix having a rank more than numbers of columns seems absurd generally.
(2) But is it possible in linear fitting or in an overdetermined system, since we have more equations than unknowns?
(3) What other conditions am I missing?
So, I am unable to answer the 3 open questions above. Kindly help me.