Probability density function of f is given as a uniform distribution, f(x)=1 and I need to find the probability distribution function of Y=X-X^2.
What I have done so far is that I found the inverse image of Y=X-X^2. Which is g^-1= 1/2 + sqrt(1/4-y). After this, I need to find probability density function of Y, which is fy(y)=fx(g^-1(y))*(derivative of g^-1(y).
My problem starts here, because derivative of g^-1(y) is not defined when y is in between 0 and 1/4. Plus, when I plot the probability density function of y that I found by myself doesn't make any sense to me. Because when I integrate the probability density function of fy(y) I cannot find 1.
I'd really appreciate any help to determine boundaries of fy(y) and/or any suggestions about how to find the inverse image of Y=X-X^2.
Sorry about the messy explanation.
I presume that $X$ is uniformly distributed over interval $\left[0,1\right]$.
Under that condition $X^{2}-X$ a.s. takes values in $\left[-\frac{1}{4},0\right]$ and for $x\in\left[-\frac{1}{4},0\right]$ we find: $$X^{2}-X\leq x\iff\frac{1}{2}-\frac{1}{2}\left(1+4x\right)^{\frac{1}{2}}\leq X\leq\frac{1}{2}+\frac{1}{2}\left(1+4x\right)^{\frac{1}{2}}$$ Denoting the CDF of $X^{2}-X$ by $F$ we find $F\left(x\right)=\left(1+4x\right)^{\frac{1}{2}}$ for $x\in\left[-\frac{1}{4},0\right]$.
We can find the PDF of $X^{2}-X$ by taking the derivative of $F$.