Assume I have
$$ X = A^TB $$
where $X \in R^{m \times n}, A \in R^{r \times m}, A \in R^{r \times n}$.
Let's define $A = [\mathbf{a}_1, ..., \mathbf{a}_m]$ where $\mathbf{a}_k $ is the $k$-th column of matrix $A$. How to compute the following derivative?
$$\frac{\partial\mathbf{a}_k }{\partial{X}_{ij}}$$
Let $$\eqalign{ C^T &= B^+ \quad &({\rm pseudoinverse\,of\,} B) \cr {\mathcal H}_{ijkl} &= \delta_{il}\delta_{jk} \quad &({\rm isotropic\,tensor}) \cr X_{ji} &= {\mathcal H}_{ijkl}X_{kl} \quad &({\rm index\,notation}) \cr X^T &= {\mathcal H}:X \quad &({\rm matrix\,notation}) \cr }$$ Your question assumes that $A$ can be expressed as a function of $X$, so let's do that.
Then we'll calculate the differential and the gradient. $$\eqalign{ A &= CX^T = C{\mathcal H}:X \cr dA &= C{\mathcal H}:dX \cr \frac{\partial A}{\partial X} &= C{\mathcal H} \cr \frac{\partial A_{ij}}{\partial X_{pq}} &= C_{ik}{\mathcal H}_{kjpq} = B^+_{ki}\delta_{kq}\delta_{jp} = B^+_{qi}\delta_{jp} \cr }$$ The result is a fourth-order tensor, and these are its component. To express it in terms of the columns of $A$ and $C$, sum over the $i$-index with vectors from the standard basis. $$\eqalign{ {\mathbf a}_j &= A_{ij}\,{\mathbf e}_i \cr {\mathbf c}_q &= C_{iq}\,{\mathbf e}_i = B^+_{qi}\,{\mathbf e}_i \cr \frac{\partial {\mathbf a}_{j}}{\partial X_{pq}} &= B^+_{qi}\delta_{jp} {\mathbf e}_i = \delta_{jp}\,{\mathbf c}_q \cr }$$