If we have, \begin{equation} Z=\begin{bmatrix} \dfrac{\partial A}{\partial x} & \dfrac{\partial A}{\partial y}\\ \dfrac{\partial B}{\partial x} & \dfrac{\partial B}{\partial y} \end{bmatrix} \end{equation}
Where $ \dfrac{\partial A}{\partial x}, \dfrac{\partial A}{\partial y}, \dfrac{\partial B}{\partial x}$ and $\dfrac{\partial B}{\partial y}$ are matrices.
Now, in order to find $\dfrac{\partial x}{\partial A}$, should I invert $Z$ and get $\dfrac{\partial x}{\partial A}$ from the inverse (if yes please tell me how, I couldn't figure it out)? or I can simply inverse the submatrix $\dfrac{\partial x}{\partial A}=\left(\dfrac{\partial A}{\partial x}\right)^{-1}$?
The question is not well posed. The OP has not specified whether $A$ is of the same dimension as $x$ and $B$ as $y$, or in other words, whether $\dfrac{\partial A}{\partial x}$ and $\dfrac{\partial B}{\partial y}$ are square matrices. Suppose the answer to the question is affirmative and in addition $\dfrac{\partial B}{\partial y}$ and $\dfrac{\partial A}{\partial x}$ are invertible. Let $a:=\dfrac{\partial A}{\partial x},\,b:=\dfrac{\partial A}{\partial y},\,c:=\dfrac{\partial B}{\partial x},\,d:=\dfrac{\partial B}{\partial y}$. By block Gaussian elimination we conclude that $\alpha:= a-bd^{-1}c$ and $\beta:= d-cd^{-1}b$ are invertible and $$Z^{-1}=\begin{bmatrix} \alpha^{-1} & -a^{-1}b\beta^{-1} \\ -d^{-1}c\alpha^{-1} & \beta^{-1} \end{bmatrix}.$$ We know $\dfrac{\partial x}{\partial A}=\alpha^{-1}$. The advantage of this algorithm is that all the inversions involved are for smaller size matrices and thus the complexity of the operation of which is greatly reduced especially for $Z$ of large dimension.