I am struggling to get my head around the likelihood function. I understand MLE in principle but where I'm falling down in practice is the very basics of formulating the likelihood function before maximising (or maximising the log). Can anyone explain to me how to formulate a likelihood function? Or point me in the direction of some good resources for this problem. I've googled it, but everything I've found has skipped over the likelihood function and dived into the MLE too quickly for me to grasp it.
Many thanks.
An effort for an informal explanation: The story is the following: you have a random variable $X$ that is distributed according to some probability density function $f_X(x)$. But you do not know some parameter of this function $f$. For example $f_X(x)=ax+b$ but you do not know $a$. So, right $f(x; a)$ instead (ok?).
Now, you start taking observations from $X$, formally you take a sample of independent random variables $X_1,X_2,\dots, X_n$ and you observe them, in order to get some information about $a$ (which you do not know, remember?). What do I mean by that: $X_1$ denotes the result (which you do not know yet) of your first sample. So, just observe it and write $X_1=x_1$ (where $x_1$ is what you saw, it is a number). To see the difference between $X_1$ and $x_1$: let's say, you forgot what you saw (you wrote it somewhere and you lost the note), then you repeat the observation and you get $X_1=x_1'$. Got it?
So, you take $n$ observations. Now, ask: What is the likelihood that I will observe $(x_1,x_2,\dots,x_n$. Answer: these observations are independent and each one comes from the distribution $f$, so (by the multiplicative law $$f(x_1,x_2,\dots x_n ; a)=f(x_1; a)f(x_2;a)\dots f(x_n;a)=\prod_{i=1}^nf(x_i;a)$$ and that is your likelihood function, or informally "the likelihood that you observed what you observed". Now, maximize this with respect to $a$. That is were your theory takes over.