How to work with operations/algebra of random variables?

Question

How are operations such as the sum, the product, the quotient, exponentiation, etc. of random variables solved or approached?

This question addresses a similar problem but starts one step further: knowing how to find the distribution of a function of random variables. I'm asking for the previous step.

Then, my question is how, in general, all the operations over random variables are handled?

Answer 1

In abstract probability based on measure theory, you are asking about the proof that any function from your list is measurable, such as $f(X,Y)=X+Y$ or $XY$ or $|X|^Y$. You can see proofs of this for specific $f$ in any textbook that discusses Lebesgue measure theory. Any continuous multivariable function is measurable, for example.

In practice $f$ is some concrete function and you need only that for any reasonable subset $S$ of the range of $f$ (an open set, a compact set, countable or "tame" unions and intersections of these), $f^{-1}(S)$ is reasonable in a similar sense and therefore "obviously measurable". Explicitly proving the measurability of a specific function would be unusual.

Answer 2

Here is a (smaller) concrete answer for the sum, ie. what is the PDF of $h(z)$:

$$ z = x + y $$

if $x$ and $y$ are drawn from independent probability distributions $f(x)$ and $g(y)$. It is very useful to realize that if $x,y$ are independent, then the function $p(x,y)=f(x)g(y)$ is the joint PDF, that is when you have two independent events the chance of both of them occurring is simply their product. Now think of the 2D plane where the point $(x,y)$ is the value $f(x)g(y)$. Integrating some area in this plane is equal to the probability of the event occouring over the area (just like in 1D).

To solve for $z=x+y$ consider all the points where $z$ is constant - this defines a line with slope $-1$ and a y-intercept of $z$. We need to get an area for this, so we can integrate the area defined by a line that is very close to it. That is, we need to find the are between the two lines:

$$ y_1 = -x + z $$ $$ y_2 = -x + z + dx $$

The integral is then:

$$ h(z) = \int_{x=-\infty}^{\infty} \int_{y=-x+z}^{y=-x+z+dx} f(x)g(y) dy dx $$ $$ h(z) = \int_{x=-\infty}^{\infty} f(x) \int_{y=-x+z}^{y=-x+z+dx} g(y) dy dx $$ $$ h(z) = \int_{x=-\infty}^{\infty} f(x)g(z-x) dx $$

There is a lot swept under the rug here and for that I'd refer you to the reference given by Kaestur Hakarl in the comments. Also, if you're physics minded (like myself) I got introduced to the subject in Arfken & Weber's Mathematical Methods for Physicists vol 6, pg 1126. Here they introduce the change in area through the Jacobian, which helps you generalize to other functions such as the product and ratio.