I am currently working myself through a paper on parameter estimation, and am trying to get a grasp on the concept of nullspace. I think I might be on to something by now, so I would like to test my understanding against the community, and hopefully get some further insight into the topic. I hope that those of you with a better grasp on the topic might be able to help me further my understanding.
Back to topic: In our case we have a linearized model, which relates an input vector $k$ to an output vector $h$ via a transformation matrix $A$. The model would look like this:
$$ h = A * k$$
where $h$ is of dimension [m x 1], $A$ is of dimension [m x n] and $k$ is of dimension [n x 1]. For parameter estimation, we want to inversely infer a vector of parameters $k$ that could reproduce measurements $h$. This is the eventual goal of the publication I am referring to.
Now the publication mentions, that as soon as the matrix $A$ has more columns than vectors (which is to say $m < n$, or more intuitively, the input vector is longer than the output vector) the transformation matrix $A$ must necessarily have a nullspace, which basically means that we cannot infer the vector $k$ uniquely, since any vector with contributions from the nullspace would yield equivalent results.
Now I have stumbled across a tutorial that explains the 'economic form' of Singular Value Decomposition (SVD), during which you basically cut out a certain part of the input or output space, because it has no effect on the calculation, and I wonder if the underlying way of thinking could be used to explain null space in the aforementioned case. I have made such a sketch myself for the case of $m < n$.
QUESTION 1: If we only look at the multiplication of the singular value diagonal matrix $S$ with the input space matrix $V^T$, we can see that there is 'dead space' (red dashes) of zeroes in the matrix $S$ beyond its diagonal entries that will always be multiplied with a certain region of the matrix $V^T$ (also red dashes). This region of the input space should hence never contribute to the total matrix product - is this the source of (or an explanation for) the aforementioned nullspace?
QUESTION 2: (Only applies if assumption in Question 1 is correct) The paper further mentions that transformation matrices with more rows than columns ($m > n$) can also sometimes have a null space (but do not necessarily have to have one). Following the logic from above, and doing the same steps, then the dead zone would affect the output space instead. The entirety of the input space would be used in that case. How would this then explain the possibility of a null space? Would this require diagonal values of zero in the $S$ matrix?
I am sorry for the potentially confusing description of the issue - as I said I am still trying to gain a grasp on the issue. I hope one of you can make sense of my rambling and shed some light on the issue.