Let's say I have a regular matrix $A \in \mathbb R^{n \times n}$, with ($b$ and $x$ are also in $\mathbb R^{n \times n}$) $$Ax=b$$
I want to find the derivative of the solution $x$ to this system of linear equations, by the entries of the matrix and the entries of $b$, so I am looking for $$\frac{\partial x}{\partial a_{ij}},\frac{\partial x}{\partial b_i}$$
I do not quite understand how to solve this problem. I am familiar with the implicit function theorem and with the inverse function theorem and I am pretty sure that these will come in handy. I tried applying the inverse function theorem, but I end up with the inverse of a vector, which cannot be quite right. What is the best way to start here? Any help is greatly appreciated!
Since $A$ is assumed to be regular, you can write $$x= A^{-1}b$$ Differentiating this with respect to $b$ (or $b_i$) is trivial, since it's linear. Differentiating the inverse of $A$ is slightly more complex, but if you note that $A^{-1} A = I$ it's quite obvious how to approach this. The result is more than well documented, so I'll just give you this link, where you will find it on page 8.