I am trying to go backwards through a neural network. I have an output and I want to see what input would lead to that output. To go forwards I start with a vector and multiply by a matrix and then multiply that resulting matrix by a vector and so on. To go back I need to divide the solution vector by a series of matrices. I tried calculating the pseudo-inverse of the matrices using Julia's built in function and multiplying but the solution was quite far off from where it should have been. Is there a way to do this more precisely? I know its not an easy problem but I would imagine there must be a way to do this with relatively small (4x7) matrices.
Edit* If my state vector is x then going forward means calculating Ax for some A.
Here's a small example to illustrate that this is not just difficult, but impossible (without more constraints/information).
Suppose that our first (column-) vector $x$ was $(3,-1,2)$. Let's take the matrix $$ A = \pmatrix{1&0&0\\0&1&0} $$ then the output $Ax$ is going to be the smaller vector $Ax = (3,-1)$.
Suppose, then, that we want to reverse this process. That is, we know that the output is $Ax = (3,-1),$ and we want to figure out which vector $x$ this came from. If we solved the associated system of equations for $x$, we would find that the solution $$ x = (3,-1,t) $$ will work for any value of $t$. If you use the pseudo-inverse, you would pick out the smallest among these answers (i.e. vector with the shortest Pythagorean length), namely $$ x = A^\dagger (Ax) = (3,-1,0) $$ which doesn't necessarily match your original input.