I am estimating a statistical model. Let $\Sigma=\beta\beta^\top + D$, where $\beta\in\mathbb{R}^p$ and $D$ is a $p\times p$ diagonal matrix. Denote $\widehat{\Sigma}$ as a consistent estimator for $\Sigma$, i.e., $\|\widehat{\Sigma}-\Sigma\| = O_p(1/\sqrt{N})$, where $N$ is the sample size and satisfying that $p\rightarrow \infty$ but $p/N\rightarrow 0$ as $N\rightarrow \infty$.
Now I am concerned about the largest eigenvalue $\widehat{\lambda}$ and its corresponding eigenvector $\widehat{v}$ of the estimated matrix $\widehat{\Sigma}$. Similarly, we define $\lambda$ as the largest eigenvalue and the corresponding eigenvector $v$ of matrix $\Sigma$. From the Weyl's inequality we have $|\widehat{\lambda} - \lambda| \leq \|\widehat{\Sigma}-\Sigma\| = O_p(1/\sqrt{N})$. However, I wanna know if there exists a faster convergence rate for $\|\widehat{v}\big/\sqrt{\widehat{\lambda}} - v/\sqrt{\lambda}\|$, such as $O_p(1/\sqrt{Np})$?
Here is a paritial answer, assuming $\hat{\Sigma}$ is also a symmetric matrix:
$$|| \frac{\hat{v}_1}{\hat{\lambda_1}^{0.5}} - \frac{v_1}{\lambda_1^{0.5}} ||_2 = || \frac{\hat{v}_1}{\hat{\lambda_1}^{0.5}} - \frac{\hat{v}_1}{\lambda_1^{0.5}} + \frac{\hat{v}_1}{\lambda_1^{0.5}} - \frac{v_1}{\lambda_1^{0.5}} ||_2 $$ $$ \leq \left | \frac{1}{\hat{\lambda_1}^{0.5}} - \frac{1}{\lambda_1^{0.5}} \right | ||\hat{v}_1||_2 + \frac{1}{|\lambda_1^{0.5}|} ||\hat{v}_1-v_1||_2$$
$$||V^T\hat{\Sigma}\hat{V} - V^T\Sigma\hat{V}||_2 = ||V^T\hat{V}\hat{D} - DV^T\hat{V}||_2 \rightarrow 0 \implies$$ $$V^T\hat{V}D \approx D V^T\hat{V}$$
$D$ is diagonal matrix. Hence $V\hat{V}^T \approx I$ assuming all eigenvalues are non-zero. Hence $\hat{v}_i \rightarrow v_i$ for all $\lambda_i \neq 0$ and $\lambda_i$ distinct with unique eigen vector.
So you need to add the condition in your question that $v_1$ is a unique eigenvector corresponding to $\lambda_1$.
From $$||V^T\hat{\Sigma}\hat{V} - V^T\Sigma\hat{V}||_2 = ||V^T\hat{V}\hat{D} - DV^T\hat{V}||_2$$, So, $\sum_i |\lambda_1-\hat{\lambda_i}|^2 \times |<v_1,\hat{v}_i>|^2 \leq p^2 ||\hat{\Sigma}-\Sigma||_2^2$.
$$||\hat{v}_1-v_1||_2 = |1-<v_1,\hat{v_1}>|^2 + \sum_i |<v_1,\hat{v_i}>|^2 \leq constant \times p^2 \times ||\hat{\Sigma}-\Sigma||_2^2 \times \frac{1}{|\lambda_1 - \lambda_2|^2}$$.
So it seems like if $\lambda_2 \approx \lambda_1$, the convergence will be slower and may not even happen as $p \rightarrow \infty$.
Try if above idea can be used !