Consider the nonlinear system of equations:
$$ F(x,\mu) = \begin{bmatrix} Ax - \mu x \\ \frac12 x^T x - 1 \end{bmatrix} $$
where $A$ is a real symmetric matrix and $\mu$ is a simple eigenvalue.
To analyze the properties of this system, we introduce a new vector $y = (x^T, \mu)^T$, where $x$ is a vector. Thus, the system can be rewritten as the equation $F(y) = 0$.
We aim to prove that, under the assumption that $μ$ is a simple eigenvalue of A, the Jacobian matrix $F'(y)$ is nonsingular.
I have calculate $$ F'(y) = \begin{bmatrix} A - \mu I & -x \\ x^T & 0 \end{bmatrix} $$ But I don't know how to prove the matrix is nonsingular. Do I need to calculate it's determinant?
Not sure this is what you want, but for eigenvalues $\mu_1,...,\mu_n$ with eigenvectors $x_1,...,x_n$ and $U=(x_1,...,x_n)$, you could use the determinant for the block matrix. Then $$\det\begin{pmatrix} A - \mu I & -x_1 \\ x_1^t & 0 \end{pmatrix} = \det(A - \mu I) \cdot x_1^t(A-\mu I)^{-1} x_1 \\ = \det(D-\mu I)\cdot e_1^t(D-\mu I)^{-1} e_1 = \prod_{k=2}^n(\mu_k-\mu)$$ where $D=U^tAU$. You can then take the limit $\mu \rightarrow \mu_1$.
So I guess this implies your matrix is non-singular, if all the eigenvalues are distinct.