This problem has been done a few times on other sites but there is no work or explanation of the steps taken to get h(x,y). I can understand getting the limits by finding the bounds of the triangle so that a = 1, b = 2, g(y) = $2-y$, and f(y) = $y-2$ based on how the triangle is drawn. However for h(x,y) the answer is $1+9y^{12}cos^3xsin^3x+36y^6cos^6x$. I found this out through guess and check based on what other answers of similar questions. Can someone please help me understand the process as to how you get this answer.
If $z=f(x,y)$ then the function $h$ you are looking for is given by $$h(x,y)=1+f_x^2(x,y)+f_y^2(x,y)\ .$$ One arrives at this result doing some three-dimensional vector geometry and a limiting process. The $h$ you obtained is wrong. Check your computations!
The requested iterated integral can be simplified a little by noting that the given surface is symmetric with respect to a reflection in the $(y,z)$-plane.