Sum of two independent random variable (convolution)

Question

I want to calculate the sum of two independent, uniform random variables. Suppose we choose independently two numbers at random from the interval $[0, 1]$ with uniform probability density. What is the density of their sum?

I need an explanation on how the interval is being set for the convolution.

Answer 1

You don't need convolution, you can see that on the spot.

Both variables have a level probability density in $[0, 1]$.

Their cross has a similarly level probability density in $[0, 1]^2$.

To get the probability density of their sum, you need to find the length of the lines $y=-x + c$ inside this 2D square:

• the result will grow linearly from 0 to 1 in $[0, 1]$
• it will decrease linearly from 1 to 0 in $[1, 2]$.

Answer 2

A convolution goes from $-\infty$ to $\infty$. Since one of them, say $Y$, has a PDF $f_Y(y)=0$ if $y\notin[0,1]$, the lower and upper limit change to $0$ and $1$ with the corresponding $f_Y(y)$. You can then perform change of variable to further evaluate the integral. I assume your r.v.'s are continuous.

Answer 3

It will be easier if you first try a discrete version, then look at it in the continuous model.

Imagine you want to add two iid RVs with discrete pdf = $ p(-1) = 0.5$ and $p(+1) = 0.5$. This is the classic 'flip a coin and heads means you win a dollar, tails means you lose a dollar.

The convolution in a discrete case is $$(f*g)[n] = \sum_{m=-\infty}^\infty f[m]g[n-m]$$ In essence, g is 'flipped bacwards' and 'slid' along, calculating this sum for every (useful) value of n.

Just to start, imagine n=3. But our pdf for both f() and g() only has values for -1 and 1. It will help you to graph what this looks like on a sheet of paper, but you are going to end up summing (and I am skipping the tails since those will never cause an overlap) $$...f(-1)g(4)+f(0)g(3)+f(1)g(2)+...$$

So, clearly, there is no overlap and you get zero, since f() is non-zero only for -1 and 1, and the same for g.

The first n for which this gives any overlap is n=2. Now, you get $$...+f(-1)g(3)+f(0)g(2)+f(1)g(1)+... $$

that last term, f(1)g(1)=$(0.5)(0.5)=.25$.

Keep going. The most overlap occurs when n=0, and the sum looks like:

$$...+f(-2)g(2)+f(-1)g(1)+f(0)g(0)+f(1)g(-1)+f(2)g(-2)...$$

Now it has the most overlap it will, because two terms both have two factors of 0.5 (f(-1)g(1) and f(1)g(-10)). The probability of earning nothing, on two tosses, is this 50% (quite simply HT or TH).

That should give you the idea. In general, if the function f() is nonzero over a range of q points, and g() is non-zero over p points, the convolution will have p+q-1 non-zero values. And, the minimum of the range will occur at the sum of the minima of the two functions (i.e., those values on the X axis that are the smallest ones that are non-zero), and the maximum of the sum of the maxima.

Note in our case, both f and g had 3 points that were not always zero outside that range (-1, 0, and 1 - although the point at 0 had zero probability) and the min was -1 for both, and the max was 1 for both. Therefore, the end result will have 5 points in the 'core' (-2, -1, 0, 1, and 2) and the min will be $-1+-1=-2$ and $1+=2$, as it does.

I suggest you try calculating $(f*g)[n]$ for $n=-1$ and $n=1$. You will see how the 'comb' makes those have no terms that aren't zero (that is kind of key to getting convolution straight in your head). An, it is right: in two tosses, you can either lose $2 (TT), win $2 (HH) - both with p=0.25 - or break even (HT or TH) with p=0.50.

It should be straightforward to generalize to continuous cases.

Wikipedia has some good static and dynamic graphics on this (look at the section 'graphical visualization'): https://en.wikipedia.org/wiki/Convolution