sum of independent exponential random variables

Question

Let $X$ be the sum of two independent exponential random variables: $X_{1}$ with parameter $\lambda_{1} = \frac{1}{5}$ and $X_{2}$ with parameter $\lambda_{2} = 2 $. These random variables have values in the interval $[0,60]$

I want to prove the variance of $X$ is $401$. I found by looking into some references that the p.d.f of $X$ is: $$ f_{X}(t) = \sum_{i=1}^{2} {\frac{\lambda_{1} \lambda_{2} }{\prod_{j=1,j \ne i}^{2}(\lambda_{j} -\lambda_{i})} exp(-\lambda_{i} t)} $$

I wanted to know if the above density is correct or not since I did find a value different than what I'm supposed to get for the variance computation.

Thanks.

Answer 1

I'm adding this as a separate answer as it is somewhat unrelated to my original post, and is quite lengthy in itself.

My previous post addresses the question originally asked, though it became clear in the comments that the question of interest is actually the following (taken from the comments):

The independent visitors of a certain Web site may be divided into two groups: those who arrived on this site voluntarily (type 1) and those who arrived there by chance or by error (type 2). Let N(t) be the total number of visitors in the interval [0,t]. We suppose that {N{t),t > 0} is a Poisson process with rate a=10 per hour, and that 80 of the visitors are of type 1 (and 20 of type 2).The question was: "Calculate the variance of the total time spent on this site by the visitors arrived in the interval [0,1] if the time (in minutes) X1 (respectively, X2) that a type 1 (resp., type 2) visitor spends on the site in question is an exponential random variable with parameter 15 (resp., 2). Moreover, we assume that both varibales are independent random variables.

I will highlight two approaches to the problem: one working with knowledge of independent variables only and Wald's equation, and the second using properties of the Poisson and Exponential distributions. Before we get to these we make the following note:

If the arrivals of a rate $\lambda$ Poisson process can be split into two types (types 1 and 2) with the proportion of type 1 arrivals being $p_1$, and type 2 occurring with $p_2 = 1 -p_1$, then the process can be `thinned' into two sub-processes with rates $\lambda_1 = \lambda p_1$, and $\lambda_2 = \lambda p_2$ respectively. These two processes are independent of each other by properties of Poisson processes. In your question, you have $\lambda = 10$ (which I interpret as an average of 10 visitors to the site per hour), and $\lambda_1 = 0.8 \times 10 = 8$, and $\lambda_2 = 0.2 \times 10 = 2$.

The question asks you to compute the variance of the total time spent by all visitors in 1 hour, where the arrivals are modeled by a Poisson process. Let $T_i$ denote the time spent by the $i$-th arrival, and $N$ the total number of arrivals in an hour, so we consider \begin{align*} T = \sum_{i=1}^N T_i. \end{align*} There are currently two problems to face: the first is that the $T_i$ are all exponential (as stated in the question), but with different parameters. The second is the fact that $N$ is itself a random variable. The solution to the first of these problems is to use our thinning relation described above: viewing the process with rate $\lambda$ as the union of the $\lambda_1$ and $\lambda_2$ processes then: $N = N_1 + N_2$, and then \begin{align*} T = \sum_{i=1}^{N_1} T_i^{(1)} + \sum_{i=1}^{N_2} T_i^{(2)}, \end{align*} where as given in the question $T_i^{(1)}$ are exponential of parameter $\tau_1$, and $T_i^{(2)}$ are exponential of parameter $\tau_2$ (as given: $\tau_1 = 2$, $\tau_2 = 1/5$). According to the formula for the variance which was given in my previous post \begin{align*} \text{Var}[T] = \text{Var}\Big[ \sum_{i=1}^{N_1} T_i^{(1)} \Big]+ \text{Var}\Big[ \sum_{i=1}^{N_2} T_i^{(2)} \Big]. \end{align*} So our problem is now about taking variances of a sum of randomly many random variables.

In the following I drop the sub-script $N_1,N_2$ since the above formula tells you how to apply this for your problem. So $N$ is now a Poisson variable of rate $\lambda$, and each $T_i$ is exponential with rate $\tau$.

The first approach is to find the variance of such a sum is to use a result called Wald's Equation, which says that under the assumption of independence of $N$ from the $T_i$, and if the $T_i$ are i.i.d. then \begin{align*} \mathbf E [ \sum_{i=1}^N T_i] = \mathbf E[N] \mathbf E[T_1] \end{align*} One can quickly extend this to a theorem about the variance instead: \begin{align*} \text{Var} [ \sum_{i=1}^N T_i] = \text{Var}[N]\mathbf E[T_1]^2 + \mathbf E[N] \text{Var}[T_1]\end{align*} You can now apply this to the case we have with $N$ a Poisson variable, independent of the $T_i$ which are i.i.d. exponential variables.

Of course, this method relies on you knowing Wald's equation, and you would probably want to be able to derive this too. So, how can we solve the problem without using that? The alternative is to use conditional expectations, and the fact that we know the distributions of the relevant variables. I'll do the calculation for the expectation of $\sum_{i=1}^N T_i$, and leave you to calculate the second moment (which will be needed to find the variance).

So if $N \sim \text{Poisson}(\lambda)$, and $T_i \sim \text{Exp}(\tau)$, then conditioning on the value $N$ takes \begin{align*} \mathbf E \Big[ \sum_{i=1}^N T_i ] & = \sum_{n=0}^N \mathbf E \Big[ \sum_{i=1}^N T_i | N = n ] \mathbf P[ N = n] \\ & = \sum_{n=0}^\infty \mathbf E \Big[ \sum_{i=1}^n T_i ] \mathbf P[ N = n] \\ & = \sum_{n = 0}^\infty n \mathbf E[\tau] \mathbf P[ N = n] \\ & = \mathbf E[\tau] e^{-\lambda} \sum_{n = 0}^\infty n \lambda^n \frac{1}{n!} \\ & = \frac{\lambda}{\tau}. \end{align*} Note that this agrees with the answer we would get from Wald's equation.

That should now be enough to let you answer the question.

Answer 2

There are a few points to be addressed in your question. First of all, exponential distributions are supported on the entirety of the positive real line, meaning that $X_1, X_2$ take values in $[0,\infty)$, rather than $[0,60]$ as you claim; moreover their sum $X = X_1 + X_2$ also takes values in $[0,\infty)$.

There are two immediate approaches to calculate the variance of $X$. The first one depends only on the fact that they are independent. A basic fact in probability theory asserts that if $U, V$ are independent random variables, then

\begin{align*} \text{Var}(U+V) &= \mathbf E[ (U+ V)^2] - \mathbf E[U + V]^2 \\ & = \mathbf E[U^2] + \mathbf E[V^2] + 2\mathbf E[U]\mathbf E[V] - (\mathbf E[U]^2 + \mathbf E[V]^2 + 2\mathbf E[U]\mathbf E[V] )\\ & = \text{Var}[U] + \text{Var}[V] \end{align*}

From this it follows from the fact that the variance of an $\text{Exp}(\lambda)$ variable is $\lambda^{-2}$, that \begin{align*} \text{Var}(X_1 + X_2) & = \lambda_1^{-2} + \lambda_2^{-2} \\ & = \frac{101}{4}. \end{align*} for $\lambda_1 = 1/5$, $\lambda_2 = 2$.

Note that in this approach we did not need any properties of the distributions, other than knowledge of their variances (i.e. if you gave me two distributions $U,V$, with $\text{Var}[U] = \lambda_1, \, \text{Var}[V] = \lambda_2$, the answer would not change).

A second approach would be to argue via the probability density functions (p.d.f.s), as you suggest in the question. This will in general be a harder approach, as we will have to derive the variance of the sum from the p.d.f., as well as computing the p.d.f.

In general for independent $U, V$, with p.d.f. $f_U, f_V$ respectively, the distribution of their sum is given as the convolution of their p.d.f.s \begin{align*} f_{U +V}(x) = \int_{-\infty}^\infty f_U(y)f_V(x-y)dy. \end{align*} In the case of a exponential distributions, the p.d.f. of a $\lambda$ exponential distribution is \begin{align*} f(x) = \lambda e^{-\lambda{x}} \end{align*} And so for two independent exponentials with parameters $ \lambda_1 \neq \lambda_2$ \begin{align*} f_{X}(x) &= f_{X_1 + X_2}(x) \\ & = \int_{0}^x \lambda_1 e^{-\lambda_1 y} \lambda_2 e^{-\lambda_2(x-y)}dy \\ & = \lambda_1 \lambda_2 e^{-\lambda_2 x}\int_{0}^x e^{(\lambda_2-\lambda_1)y}dy \\ & = \frac{\lambda_1 \lambda_2}{\lambda_2-\lambda_1}( e^{-\lambda_1 x}-e^{-\lambda_2 x}). \end{align*} Note that we only integrate over the region supported by the p.d.f.s. From this we can then proceed to calculate the variance from the p.d.f. by the standard formulae \begin{align*} \mathbf E[V] &= \int_{-\infty}^\infty x f_V(x) d x,\\ \mathbf E[V^2] &= \int_{-\infty}^\infty x^2 f_V(x) dx \end{align*}

In all, I hope that my post has made it clear that this second approach requires a lot more work, and is more prone to calculation error!