A point $(X, Y)$ is randomly selected from the interior of a square with vertices $(0, 0),(3, 0),(3, 3),(0, 3)$. $Z$ in an area of a rectangle with vertices $(0, 0), (X, 0), (X, Y), (0, Y)$. Determine the distribution of the variable $Z$, expected value of $Z$ and variance of $Z$.
As I understand, we know that there are two random variables. One is $X$, which follows a uniform distribution on the interval $[0,3]$ and the other one is $Y$. It follows a uniform distribution on the same interval, that is $[0,3]$. The two random variables are independent. There is also a rectangle constructed for which the lengths of two adjacent sides are $X$ and $Y$.
We have a following picture - we choose a random point $(X,Y)$ somewhere in a red square:
Therefore we know that $Z = XY$, so:
PDFs of a $X$ and $Y$ with uniform distribution is equal to $\frac{1}{3-0} = \frac{1}{3}$ for $x, y \in [0,3]$ $0$ otherwise.
Therefore PDF of Z is equal to $\frac{1}{3} \cdot \frac{1}{3} = \frac{1}{9}$ for $z \in [0,3]$ and $0$ otherwise. PDF is unique for a distribution therefore we have our distribution.
$$E(Z) = E(XY)$$ $$E(Z) = E(X) + E(Y) + Cov(X,Y)$$ We know that variables X and Y are independent, therefore $Cov(X,Y) = 0$ and we have: $$E(Z) = E(X) + E(Y)$$ $E(X) = E(Y) = \frac{0 + 3}{2} = \frac{3}{2}$ and we get that: $$E(Z) = \frac{3}{2} + \frac{3}{2} = 3$$
I don't know how to calculate variance of $Z$.
Your distribution of $Z = XY$ is wrong. This is a common mistake: the distribution of the product of two random variables is not just the product of their pdfs. You have to formally derive it and it is surprisingly involved for such a simple setup:
Since the support of both $X, Y$ is $(0, 3)$, the support of $Z = XY$ will be $(0, 9)$; so for $z \in (0, 9)$ \begin{align*} F_Z(z) = P[XY \leq z] = P\Big[Y \leq \frac{z}{X}\Big] &= \int\int_{\big\{(x, y) \in [0, 3]^2 \ : \ y \leq z/x \big\}} f_{X, Y}(x, y) \ dydx \\ &= \frac{1}{9}\int_0^{z/3}\int_0^3 \ dydx + \frac{1}{9}\int_{z/3}^3 \int_0^{z/x} \ dydx \\ &= \frac{z}{9} + \frac{z (\log(9) - \log(z))}{9} = \frac{z}{9}\Big[1 - \log\Big(\frac{z}{9}\Big)\Big] \end{align*} and $F_Z(z) = 0, z \leq 0$ and $F_Z(z) = 1, z \geq 9$. Differentiating this cdf we get that $$ f_Z(z) = -\frac{1}{9}\log\Big(\frac{z}{9}\Big) \quad 0 < z < 9 $$ and $0$ elsewhere.
For the expectation and variance, your reasoning, using independence, for the expectation is almost correct. You got $E[Z] = E[X] + E[Y]$ which is a silly mistake because $Z = XY$ was the product not the sum of those variables. So naturally it should instead be $$ E[Z] = \underbrace{ E[XY] = E[X]E[Y] }_{\text{by independence}} = \frac{9}{4} $$ Finally for the variance, you can use the pdf given above or you can use $\text{Var}[Z] = E[Z^2] - E[Z]^2$ along with independence to note that $E[Z^2] = E[X^2Y^2] = E[X^2]E[Y^2]$.