Let $E$ be the intersection of the cylinders $x^{2}+y^{2} \leq 1, y^{2}+z^{2} \leq 1$.
Compute flux $ \ \iint_{\partial E} F \cdot d S$
where $\vec F = \left(x y^{2}+\cos (y z)\right) \hat i - \left(x^{2} + \sin (z x)\right) \hat j + (z + \cos (x y)) \hat k \ $ and $ \ \partial E$ is oriented outward.
What I tried:
Image2:
Please guide have I got it right.
Your approach is correct and your working is correct except one mistake that led to wrong answer.
$\nabla \cdot \vec F = y^2 - 0 + 1 = y^2 + 1$ as you found.
So you are right that the volume integral to find flux is,
$\displaystyle \iiint_E (1+y^2) \ dV \ $, where $E$ is a Steinmetz solid made by intersection of two cylinders $x^2 + y^2 \leq 1$ and $y^2+z^2 \leq 1$. Using volume of thin cylindrical slices perpendicular to $y$ axis, the volume integral becomes,
$\displaystyle \iiint_V (1+y^2) \ dV \ = \displaystyle 4 \int_{-1}^1 (1+y^2) (1-y^2) \ dy = \displaystyle 8 \int_0^1 \color {blue} {(1-y^4)} \ dy$
Your mistake: You wrote the integrand as $(1-y^2)$ instead of $(1-y^4)$