I want to prove
$(d\exp_x)_{p}$ is singular iff there exists a normal Jacobi field $U(t)$ along $\gamma(t)=\exp_x(tp)$ not identically zero such that $U(0)=U(1)=0$.
I have question about $(\implies)$ direction. By normal Jacobi field, I mean $\langle U(t),\gamma'(t)\rangle=0$.
Since $(d\exp_x)_p$ is nondegenerate in the direction of $p$. Therefore we only need to consider $X$ s.t. $\langle X,p\rangle=0$
Let $X\in T_xM\simeq T_p(T_xM)$ s.t. $\langle X,p\rangle=0$ Then $\gamma_u(t)=\exp_x[t(p+uX)]$ gives a normal Jacobi field with $U(0)=0,U'(0)=X, U(1)=(d\exp_x)_p(X)$.
Therefor if $X\in \ker((d\exp_x)_p)$ and $X\not=0$ then $U(t)$ is the required normal Jacobi field.
But I am not sure whether such $X$ exists. That is given $p\not\in \ker((d\exp_x)_p)$, find $X\in \ker((d\exp_x)_p)$ such that $X\not=0, \langle X,p\rangle=0$. I know if $\dim \ker((d\exp_x)_p)>1$, it's possible. But I am not sure how to handle $\dim \ker((d\exp_x)_p)=1$. Is there any property of $d\exp_x$ that I should use?
The Gauss lemma states that for $X\in T_xM$ we have$$\langle d(\exp_x)_p(X),d(\exp_x)_p(p)\rangle=\langle X,p\rangle.$$It follows that the kernel of $d(\exp_x)_p$ is contained in the orthogonal complement of $p$, as you conjectured.