If the population estimators follow a given relationship , can we assume that the sample estimators would follow the same relationship as well?

24 Views Asked by At

Say in case of a standard CLRM ( classical linear regression model ) we are aware that the population estimaotrs $\beta_1$ , $\beta_2$ etc. satisfy the following relation that

$f(\beta_1$,$\beta_2$,$\beta_3$ ...) = 0 $

Does this imply that the standard CLRM sample estimators i.e. $\beta'$ ,$\beta_2'$ etc. will follow the same relationship , i.e.

$f( \beta_1'$,$\beta_2'$ ...) = 0 $

I could deduce that in case of simple linear relationships we could expect this to be the case for simple linear relationship by taking expectation and using the property that $E(\beta_1'+\beta_2' +....)$ = $E(\beta_1') + E(\beta_2') +...$

But I am not so sure if I could follow the same procedure with a more general case.

1

There are 1 best solutions below

0
On

Without saying something about the structure or the properties of $f$ it is hard to answer your question. Moreover, it is worth to distinguish between $\mathbb{E}f(\hat{\beta_1},...,\hat{\beta_p})$, $f(\mathbb{E}\hat{\beta_1},...,\mathbb{E}\hat{\beta_p})$ and $f(\hat{\beta_1},...,\hat{\beta_p})$. I.e., the first two structures will be equal for linear $f$s and would probably differ for every non-linear $f$ (Jensen inequality). While, the last one will always differ from these two. This is due to the fact that, for the classical settings of linear model, the joint distribution of the estimated $\beta$ vector is Multivariate normal distribution. Thus for any non-trivial or non constant $f$ there is no reason to expect the equality to hold.

E.g., assume that $f(\beta_1,...,\beta_p)=\sum_{i=1}^p\beta_i=0$, in this case you can use the fact that your estimators are unbiased, hence $$ \mathbb{E}\sum_{i=1}^p\hat{\beta}_i=\sum_{i=1}^p\mathbb{E}\hat{\beta}_i=\sum_{i=1}^p{\beta}_i=0, $$ however $\mathbb{P}(\sum_{i=1}^p\hat{\beta}_i=0)=0$.

Now assume that $f(\beta_1,...,\beta_p)=\prod_{i=1}^p\beta_i=0$, in this case, for non centralized design matrix $X$, the estimator are correlated and thus only the second structure holds, i.e., $f(\mathbb{E}\hat{\beta_1},...,\mathbb{E}\hat{\beta_p})=\prod_{i=1}^p\mathbb{E}\hat{\beta}_i=0$.