I want to compute the derivative $$ \frac{d(A^TAh-A^Tt)}{dA} $$ and by doing some quick back of the envelope calculations I get $$ (A+A^T)h-t $$ but this does not agree with another result I got previously (and also seems strange from the perspective of dimensions). I did not find any result that I could easily adapt from a Google search, and I would like to avoid having to dig up some old linear algebra textbook. If somebody could point me in the right direction it would be great.
Moreover, what would happen with
$$ \frac{d(A^+(Ah-t))}{dA} $$
where $A^+$ is the pseudo-inverse? How are these two related and when can they be equal?
Let $f:A\in M_n\mapsto A^TAh-A^Tt$. Its derivative is
$Df_A:H\in M_n\mapsto H^TAh+A^THh-H^Tt$.
Let $g:A\in M_n\mapsto A^+(Ah-t)$. If $rank(A^+)$ is locally constant, then its derivative is
$Dg_A:H\in M_n\mapsto (-A^+HA^++A^T{A^+}^TH^T(I-AA^+)+$
$(I-A^+A)H^T{A^+}^TA^+)(Ah-t)+A^+Hh$.