I have two hypothesis as below:
Under $H_1$, $f_x(x) = 3/2 * x^2$ where $x \epsilon (-1,1)$
Under $H_0$, $f_x(x) =$ Uniformly distributed between $x \epsilon (-1,1)$
What is the maximum likelihood decision rule to maximize the detection where false alarm (given that $H_0$ happened but $H_1$ decided) is less than or equal to 0.1?
How can i solve this?
Regards...
My interpretation of the Problem: I suppose you are using a single observation, and testing $H_0: f_0(x) = .5,$ for $-1 < x < 1$ against $H_1: f_1(x) = \frac 3 2 x^2$ for $-1 < x < 1,$ at the significance level $\alpha = .1.$ (It is customary to state $H_0$ first.)
Main idea: According to the LR test you want the rejection region to be where $f_0$ is small relative to $f_1.$ So the rejection region will be in two parts near $-1$ and $1$ such that, under the density $f_0(x),$ the total probability is $\alpha = .1.$
In the sketch below, you want the probabilities outside the vertical dotted blue lines and under the density $f_0$ (green) to add to $\alpha = .1.$