NOTE: I have asked this question on stat.stackoverflow but got no answers/comments. Hence I decide to ask it on the math.stackoverflow platform as well.
I'm studying IV estimation by myself and have some confusion about the basics. Let $y=X\beta_0 + u$ be a linear model with endogenous variable $X$, and $Z$ be an instrument, meaning that $Z$ and $u$ are uncorrelated, and $Z$ and $X$ are correlated. For simplicity let's say both $X$ and $Z$ are univariate.
The IV estimation takes the form of $$ \widehat\beta = \beta_0 + (Z'X)^{-1}Z'u. $$
I understand that $\widehat\beta$ is consistent because $Z'u\overset{p}{\to} 0$, $Z'X\overset{p}{\to} a\neq 0$ and therefore by Slutsky's theorem $\widehat\beta\overset{p}{\to}\beta_0$. However, I'm having some trouble on other properties of $\widehat\beta$.
- Is $\widehat\beta$ unbiased? For me it seems $\widehat\beta$ is not unbiased, because $Z'u$ and $Z'X$ are correlated (are they?) and therefore $\mathbb E[Z'u]=0$ does not mean $\mathbb E[(Z'X)^{-1}Z'u]=0$?
- If $\widehat\beta$ is indeed biased, how do we obtain the (asymptotic) variance of $\widehat\beta$? If we calculate by definition then $\mathbb E|\widehat\beta-\beta_0|^2 = \mathbb E[(Z'X)^{-1}Z'uu'Z(Z'X)^{-1}]$ but $u$ is not uncorrelated with $(Z'X)^{-1}Z'$, and how do we simplify this expression?
1) Yes, $\hat{\beta}$ is indeed biased:
$$E[\hat{\beta}] = \beta_{0} + E_{Z, X, u}[(Z'X)^{-1}Z'u] = \beta_{0} + E_{Z, X}[(Z'X)^{-1}Z'E(u\mid Z, X)]$$
via the law of iterated expectations. In order to have $\hat{\beta}$ unbiased we should have $E(u\mid Z, X)$ = $0$, but this is an assumption too strong since it would also imply $E(u\mid X)$ = $0$, case in which you would not even have an endogeneity problem with OLS estimators.
2) For the asymptotic variance of $\hat{\beta}$ consider that: $$\hat{\beta} = \beta_{0} + (Z'X)^{-1}Z'u = \beta_{0} + (n^{-1}Z'X)^{-1}n^{-1}Z'u $$
From which:
$$\sqrt{n} (\hat{\beta} - \beta_{0}) = \left ( \frac{1}{n}Z'X \right )^{-1}\frac{1}{\sqrt{n}}Z'u$$
and since we're assuming $plim\left ( \frac{1}{n}Z'X \right )$ $\neq$ $0$ and $plim \left ( \frac{1}{n}Z'u \right )$ = $0$, then the plim of left hand side converges to $0$.
In particular, define $plim\left ( \frac{1}{n}Z'X \right )$ = $Q_{ZX}$, $plim\left ( \frac{1}{n}X'Z \right )$ = $Q_{XZ}$ and $plim\left ( \frac{1}{n}Z'Z \right )$ = $Q_{ZZ}$.
At this point, I've worked out the solution as follows:
\begin{align} As.Var(\hat{\beta}) &= \frac{1}{n} plim E \left [ \left ( \frac{1}{n}Z'X \right ) ^{-1}\left (\frac{1}{\sqrt{n}}Z'u\right )\left (\frac{1}{\sqrt{n}}u'Z \right ) \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ & = \frac{1}{n} E \left [ plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{\sqrt{n}}Z'u\right )plim \left (\frac{1}{\sqrt{n}}u'Z \right ) plim \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ & = \frac{1}{n} E \left [ plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{n}Z'uu'Z \right ) plim \left ( \frac{1}{n}X'Z \right ) ^{-1} \right ] \\ &= \frac{1}{n} \left [plim \left ( \frac{1}{n}Z'X \right ) ^{-1} plim \left (\frac{1}{n}Z'E(uu')Z \right ) plim \left (\frac{1}{n}X'Z \right )^{-1} \right ] \\ &= \frac{1}{n} \sigma^{2}Q_{ZX}^{-1}Q_{ZZ}Q_{XZ}^{-1} \\ \end{align}
where we have used homoskedasticity, plim properties and Central Limit Theorem (explaining the $\frac{1}{n}$)