The Jacobian matrix $\mathbf J$ is defined by $$\mathbf J= \begin{pmatrix} \dfrac{\partial y_1}{\partial \mu_1} & \cdots & \dfrac{\partial y_1}{\partial \mu_k}\\ \vdots & \ddots & \vdots\\ \dfrac{\partial y_k}{\partial \mu_1} & \cdots & \dfrac{\partial y_k}{\partial \mu_k} \end{pmatrix}.$$
How do I find $\mathbf J ^ {-1}$ if it exists? Is there a way to find it without differentiating each function?
I guess that you want to calculate the Jacobi matrix of coordinates change form $(y_i)$ to $(\mu_i)$ i.e. matrix of derivative $D(y(\mu))$. Since coordinate change is invertible, you can first invert coordinates back, from $(\mu_i)$ to $(y_i)$, and then calculate a matrix $D(\mu(y))$. If we denote $J = D(y(\mu))$ it follows that $J^{-1}= D(\mu(y))$, because $$ D(y(\mu)) \circ D(\mu(y)) = D(E) = E $$ where $E$ is identity matrix.