Linear regression as $\dim(\beta) \rightarrow \infty$

Question

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is given and $\beta$ is unknown. Then, $$ \beta =(E[X_i'X_i])^{-1}E[X_i'Y_i] $$ I am wondering what can be said about $\beta$ as $K \rightarrow \infty$ i.e. the dimension of $\beta$ approaches infinity? Specifically, is $\beta$ unique? Can $\beta$ be expressed in terms of functionals of the joint distribution of $(X_i,Y_i)$? Is there a way to estimate $\beta_k$ for each $k$? etc.

Answer 1

If your sample size remains constant, you will have $k>n$ where $i=1,...,n$ so your matrix will not be invertible and an infinite (continuum) betas will be able to minimize the square errors.