Let's say I have a matrix of data that is fully populated except for one value. For example, Column 1 is Height, Column 2 is Weight, Column 3 is Years Weight Lifting, and Column 4 is Bench Press Weight. So I surveyed 20 people and got their height, weight, years lifting, and bench press weight. Now I have a 5'11 individual weighing 170 pounds who has lifted for 1 year, and would like to predict his/her bench press weight. You could look at this as the matrix having a missing value, or you could look at it as wanting to predict a dependent variable given a vector of independent variables. There are curve fitting approaches to this kind of problem, but I would like to know how to use the Singular Value Decomposition to answer this question, if that is possible.
I am aware of the Singular Value Decomposition as a means of predicting missing values, but virtually all the information I have found has been in relation to huge, highly sparse matrices, with respect to the Netflix Prize and related problems. I cannot figure out how to use SVD or a similar approach to predict a missing value from a small or medium sized, fully populated (except for one missing value) matrix.
A step-by-step approach for solving the example above using SVD would be very helpful to me. Thank you!
Take a look at this paper. The basic idea is to solve the following optimization problem:
$$ \min \|X\|_*\\ \text{such that } X_{ij}=M_{ij}\quad \forall (i,j)\in\Omega $$ Here $X$ is the unknown matrix, $M$ is the known matrix, and $\Omega$ is the set of indicies known (in your case, $\Omega$ is all but one index). The special norm $\|X\|_*$ is the Nuclear Norm, defined using singular values as
$$ \|X\|_*=\sum_{j=1}^k\sigma_j $$ Here $k$ is the minimum dimension of your matrix. This optimization problem is convex, and the linked paper discusses solution details. Hope this helps!