Consider a symmetric matrix $H(t)$ parametrized smoothly by $t\in\mathbb R$. Suppose there are orthonormal eigenvectors $\phi_i(t)$ with corresponding eigenvalues $\lambda_i(t)$: $$ H(t)\phi_i(t)=\lambda_i(t)\phi_i(t). \tag{1} $$ Given $H(0)$ and $H'(0)$, I would like to find $\phi_i'(0)$. If $H(0)$ has degenerate eigenvalues, I do not see how to find all components of the eigenfunctions corresponding to a degenerate eigenvalue at $t=0$. Let me elaborate. $\newcommand{\ip}[2]{\langle#1,#2\rangle}$
Let me drop the argument $(t)$ whenever $t=0$. First, the eigenvectors are normalized, yielding $$ \ip{\phi_i'}{\phi_i}=0. \tag{2} $$ Differentiating (1) with respect to $t$ at $t=0$ gives $$ H'\phi_i +H\phi_i' = \lambda_i'\phi_i +\lambda_i\phi_i'. \tag{3} $$ Taking inner product with $\phi_i$ and using symmetry and (2) gives $$ \lambda_i'=\ip{\phi_i}{H'\phi_i}. \tag{4} $$ These are the first order perturbations of the eigenvalues.
I also want to find the first order perturbation of the eigenvectors, $\phi_i'$. Doing so is equivalent with finding the inner product $\ip{\phi_j}{\phi_i'}$ for all $j$.
Taking inner product of (3) with $\phi_j$ for $j\neq i$ gives $$ \ip{\phi_j}{H'\phi_i} = (\lambda_i-\lambda_j)\ip{\phi_j}{\phi_i'}. $$ Thus if $\lambda_j\neq\lambda_i$, we obtain $$ \ip{\phi_j}{\phi_i'} = \frac{\ip{\phi_j}{H'\phi_i}}{\lambda_i-\lambda_j}. \tag{5} $$ On the other hand, if $\lambda_j=\lambda_i$, we obtain $\ip{\phi_j}{H'\phi_i}=0$. That is, the block of $H'$ corresponding to the eigenspace of $H$ of eigenvalue $\lambda_i$ is diagonal. We have freedom in choosing the orthonormal basis when the spectrum degenerates, and it must be chosen in this way to make $\phi_i(t)$ depend continuously on $t$. Therefore the diagonal values in (4) are in fact eigenvalues of this block matrix.
For $i=j$ the desired inner product $\ip{\phi_j}{\phi_i'}$ is given by (2). If $i\neq j$ and $\lambda_i\neq\lambda_j$, it is given by (5). But what is the inner product $\ip{\phi_j}{\phi_i'}$ when $i\neq j$ and $\lambda_i=\lambda_j$?
It seems to me that the presented calculations give no constraints. I have lost no information by taking the inner product of (3) with all $\phi_j$ and looking at all the inner products instead of the vector equation. I must be missing something.
(This puzzled me for quite a while and I found no answer elsewhere, so I decided to share the question and an answer here. Actually, I figured it out while writing this question. Other answers are very much welcome!)
The given data does not determine the inner product $\newcommand{\ip}[2]{\langle#1,#2\rangle}\ip{\phi_j}{\phi_i'}$ when $i\neq j$ and $\lambda_i=\lambda_j$.
For a concrete example, let $$ H(t) = \begin{pmatrix} 2t&t^2\\t^2&0 \end{pmatrix}. $$ This has been chosen so that $H(0)=0$ is degenerate and $H'(0)$ is already diagonal. The normalized eigenvector (unique up to sign) corresponding to the eigenvalue $t(1-\sqrt{1+t^2})$ is $$ \frac1{\sqrt2} \begin{pmatrix} -\sqrt{1-\frac1{\sqrt{1+t^2}}} \\ \sqrt{1+\frac1{\sqrt{1+t^2}}} \end{pmatrix}. $$ The lower component is $1+O(t^2)$ and the upper one is $t/2+O(t^3)$.
However, if you take $H(t)=\operatorname{diag}(2t,0)$, the data $H(0),H'(0)$ is the same and $\phi_i(0)$ are also unchanged. However, $\phi_i'(0)$ does change: the eigenvectors are independent of $t$ now. (The inner product $\ip{\phi_i}{\phi_i'}$ is zero in both cases, the only difference is in the direction orthogonal to $\phi_i$.) The missing inner products depend on higher order derivatives of $H(t)$ at $t=0$.
The desired components can be found using second order perturbation theory, at least when $H'(0)$ is non-degenerate. Take two distinct indices $i$ and $j$ so that $\lambda_i=\lambda_j$. We also assume that $\lambda_i'\neq\lambda_j'$; this is the mentioned non-degeneracy assumption. Using the observations in the question, we find $H'\phi_j=\lambda_j'\phi_j$. This becomes useful once we differentiate (1) twice at $t=0$ and take an inner product with $\phi_j$. We find $$ \ip{\phi_j}{\phi_i'} = \frac{\ip{\phi_j}{H''\phi_i}}{2(\lambda_i'-\lambda_j')}. $$ The details are left as an exercise. If $H'$ is degenerate, I assume higher order derivatives are needed.