I am currently working on writing some automatic differentation code which can compute the derivative of matrix equations. I began with a simple linear matrix multiplication (A is singular):
$$A\cdot B=C$$
To be specific I formulated my cost function to be:
$$\text{Cost} = e^T \left[(C-T)\circ(C-T)\right] e$$
which equals the squard sum of the difference between the values in $C$ and a target matrix $T$.
Assuming $A,C $ and $T$ are known, I would like to find the matrix $B$.
I assumed it was somewhat familiar to a least squares problem so I tried the following:
$$A^TA\cdot B=A^TT$$
This would be a solution to minimise the eucledian norm of $A\cdot B - T$ if $b$ and $T$ were vectors. I am not sure if this can be applied to matrices and even if it could, how one could proceed. I am aware that there will exist more than one solution.
Using concrete numbers, I was using:
$$A=\begin{pmatrix}0 & 1 & 0 & 1 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\1 & 1 & 0 & 1 \end{pmatrix}, T=\begin{pmatrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\0 & 0 & 0 & 1 \end{pmatrix}$$
My aproach using gradients converged to:
$$ B=\begin{pmatrix}-0.33333 & 0.66667 & 0 & 0.33333 \\ 0.94686 & -0.84842 & -0.29708 & -0.30286 \\ 0 & 0 & 1.00000 & 0 \\-0.28020 & 0.51509 & 0.29708 & 0.63620 \end{pmatrix} $$
With
$$ C=\begin{pmatrix}\frac{2}{3} & -\frac{1}{3} & 0 & \frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & 0 & \frac{1}{3} \\ 0 & 0 & 1 & 0 \\\frac{1}{3} & \frac{1}{3} & 0 & \frac{2}{3}\end{pmatrix} $$
What solution is there to compute any of the optimal solutions? The Cost with my aproach is equal to $1$
Greetings Finn