$$\int_{0}^{1}\,dx \int_{0}^{1}\frac{x-y}{(x+y)^{3}}\,dy$$
My maths teacher gave this question to explain that if you change $dy$ with $dx$ the integral will have different value. My opinion is that the integral should come out as equal because this integral represents volume under a curve and the area over which we are integrating is same in both cases. Moreover this volume has to be $0$ because I plotted this curve in a 3d graph plotter and the surface is positive on one side of line $x=y$ and equally negative on the other side. BUT! after integrating the answer is coming as $1/2$ in one case and $-1/2$ in the other. Please explain.
I'm afraid your teacher is right. If we integrate over $x$ first,$$\int_0^1[(x+y)^{-2}-2y(x+y)^{-3}]dx=[y(x+y)^{-2}-(x+y)^{-1}]_0^1=-(1+y)^{-2}$$implies the result is $[(1+y)^{-1}]_0^1=-\frac12$. But if we integrate over $y$ first,$$\int_0^1[2x(x+y)^{-3}-(x+y)^{-2}]dy=[(x+y)^{-1}-x(x+y)^{-2}]_0^1=(x+1)^{-2}$$implies the result is $[-(1+x)^{-1}]_0^1=\frac12$.
Let's consider the geometry of this. The surface $z=\frac{x-y}{(x+y)^3}$ has parts with positive and negative $z$ for $(x,\,y)\in[0,\,1]^2$, and your hope was to define a net (above minus below) volume within this region for $x,\,y$. But the result is undefined, because the part-volumes above and below $z=0$ are each infinite, and $\infty-\infty$ is an indeterminate form. Simpler examples of this phenomenon (by which I mean they don't involve double integrals, but Wikipedia offers something similar that does) include $\int_{-a}^b\tfrac1xdx$ with $a,\,b>0$. As @CharlesHudgins notes, infinite series do this too.