My question is whether there is some connection between fitting probability distribution on some data set and linear regression? Or this two tools are for different problems?
By fitting probability distribution I mean that I have some data $x_{1},...,x_{n}$ and I believe they came from for example normally distributed population with parameters $\mu$ and $\sigma^2$. And I will estimate these parameters with likelihood estimation technique or method of moments technique.
By regression, I mean that I have random variable $X$ that is modelled for example as $X \sim a+bY+cZ$ where $Y$ and $Z$ are some random variables and I am estimating parameters $a,b$ and $c$.
So my question is whether there is some link between these two "mathematical topics"? Whether for example, I can look on fitting probability distribution as some specific problem of linear regression?
Thank you for your answer.
Yes, if you assume that your model is $$ Y_i = \beta_0 + \beta_1 x_i + \epsilon_i, $$ where $ \epsilon_i|X \sim \mathcal{N}(0, \sigma^2) $. Hence, fitting a regression model is the same as estimating the parameters of the (conditional) distribution of $y_i$. I.e., you assume - by imposing the distribution of the noise term $\epsilon_i$ - that $$ y_i |X \sim \mathcal{N}(\beta_0 +\beta_1x_i, \sigma^2), $$ thus estimating the coefficients $\beta_0$ and $\beta_1$ and the variance $\sigma^2$ is the same as estimating the conditional expectation and variance of $y|X$.
Moreover, in a process of model selection - selecting a model from a set of possible models is the same as fitting a distribution to $y|X$. Namely, from statistical POV fitting probability model and regression analysis are very closely related topics.