Suppose $X_1, X_2 \sim U(0,1)$ and $Y = \min(X_1, X_2), Z = \max(X_1, X_2)$. Find $E[Y^2]$. I want to solve this in 2 particular ways, both using the law of unconscious statistician, but with a different $f_Y(y)$.
We know $P(Y \leq y, Z \leq z) = z^2 - (z - y)^2$ (if you visualize this on a 2D plot, this should become more obvious). From this we can get
$$ f_Y(y) = \frac{\partial}{\partial y} \left( P(Y \leq y, Z \leq z) \right) = 2 (z - y) $$
So here we have $f_Y(y)$ as a function of $z$.
Another way to get $f_Y(y)$ is from order statistics for IID uniform random variables, where we get
$$ f_Y(y) = 2 (1 - x) $$
Here $f_Y(y)$ is a function of $x$.
From the LOTUS, we have $$ E[Y^2] = \int_0^1 y^2 f_Y(y) dy $$
I'm a bit confused how this integration will work with the above two $f_Y(y)$. The expectation should give a numerical value, but integrating with the above $f_Y(y)$ will yield an expectation as function of either $z$ or $y$. What am I doing wrong here?
The marginal density of $Y$ must be a function of the realization $y$ alone, and no other random variables. Specifically, $$\begin{align} \Pr[Y > y] &= \Pr[\min(X_1, X_2) > y] \\ &= \Pr[(X_1 > y) \cap (X_2 > y)] \\ &\overset{\text{ind}}{=} \Pr[X_1 > y]\Pr[X_2 > y] \\ &= (1 - y)(1-y) \\ &= (1-y)^2, \quad 0 \le y \le 1. \end{align}$$ Consequently, $$\Pr[Y \le y] = 1 - \Pr[Y > y] = 1 - (1-y)^2, \quad 0 \le y \le 1,$$ and $$f_Y(y) = 2(1-y), \quad 0 \le y \le 1.$$
it doesn't make sense to write $$f_Y(y) = 2(1-x)$$ since you have not specified what $x$ means.
It also doesn't make sense to write $f_Y(y) = 2(z-y)$ because as I have stated, the marginal cannot depend on any other random variables. The correct idea here is $$f_Y(y) = \int_{z=0}^1 f_{Y,Z}(y,z) \, dz,$$ that is to say, you are integrating the joint density of $(Y,Z)$ over the support of $Z$. Indeed, if we are to "trust" your formula, it would make more sense to write $$f_Y(y) = \frac{\partial}{\partial y}[\Pr[Y \le y, Z \le 1]],$$ not $Z \le z$.
In any case, once we know $f_Y(y) = 2(1-y)$, the calculation of the desired expectation is straightforward.