Exponential Likelihood Function

Question

Suppose $X_1, ..., X_n \stackrel{iid}{\sim}$ Exponential(rate = $\lambda$) independent of $Y_1, ..., Y_n \stackrel{iid}{\sim}$ Exponential$(1)$.

Define $Z_i \equiv \min\{X_i, Y_i\}$

I want to find the maximum likelihood estimator for $\lambda$ in the following scenario: I observe $Z_1, ..., Z_n$ and $Y_1, ..., Y_n$ but NOT any of the $X_i$.

First I need to determine the likelihood and then maximize it over $\theta > 0$, but I'm not really sure of the right approach. I calculate the joint cdf as follows:

$$P(Z_i \leq z, Y_i \leq y) = \begin{cases} P(Y_i \leq y), & y \leq z \\ P(Y_i \leq z, Y_i \leq X_i) + P(Y_i \leq y, X_i \leq z, X_i < Y_i), & y > z\end{cases} \\ = \begin{cases} 1- e^{-y}, & y \leq z \\ 1-e^{-z} + (e^{-z}-e^{-y})(1-e^{-\lambda z}), & y > z \end{cases}$$

This is because $Z_i \leq Y_i$ always. Would the likelihood function therefore be:

$$L(\lambda |Y_i, Z_i, i \in \{1,...n\}) = \prod_{\{i : Y_i = Z_i\}} (1-e^{-Y_i}) \prod_{\{i:Y_i > Z_i\}} \lambda e^{-Y_i}e^{-\lambda Z_i}$$

splitting into the "discrete" and "continuous" parts? Or am I getting this wrong? Or should I be doing something like here or here? I should note my scenario is different than theirs, as intuitively at least, observing the magnitude of the difference between the minimum and the maximum (in the cases where $Z_i$ and $Y_i$ differ) should give us more information about $\lambda$, right?

Answer 1

Would this be $$\prod_{\{i: Y_i = Z_i\}} \frac{1}{\lambda +1} \prod_{\{i: Y_i > Z_i\}} e^{-Y_i}\lambda e^{-\lambda Z_i} $$

where we just have the point mass/probability of equality contributing when $Y_i = Z_i$ and the joint density contributing otherwise. Can someone please provide some insight?

Answer 2

I would guess that the useful information is in the values of $Z_i$ and how often $Y_i=Z_i$ or not (perhaps call this $Q$); the actual values of $Y_i$ may not help beyond this.

I think you could show $Z_1, ..., Z_n \stackrel{iid}{\sim} \text{ Exponential(rate }= \lambda+1)$ and independently $Q \sim \text{ Binomial}\left(n,\frac{1}{\lambda+1}\right)$. In that case the useful likelihood of observing $z_1,\ldots,z_n$ and $q$ (so ignoring parts related to $Y_i-Z_i$ when that is positive) would be proportional to

$$(\lambda+1)^ne^{-\sum(\lambda+1) z_i} {n \choose q}\frac{\lambda^{n-q}}{(\lambda+1)^n}={n \choose q} \lambda^{n-q} e^{-(\lambda+1)\sum z_i}$$

with logarithm a constant plus $$(n-q) \log(\lambda) -(\lambda+1)\sum z_i$$

and derivative of the logarithm with respect to $\lambda$ $$\frac{n-q}{\lambda} - \sum z_i$$

and the maximum likelihood estimator $$\hat \lambda = \frac{n-q}{\sum z_i}$$

Answer 3

If you observe both $Z_i$ and $Y_i$, then when they are equal, you know $X_i > Y_i$. When they are not, you know $X_i = Z_i$. Therefore, your likelihood function is $$\begin{align*}\mathcal L(\lambda \mid \boldsymbol z, \boldsymbol y) &= \prod_{i=1}^n \left(f_X(z_i) \mathbb 1 (z_i \ne y_i) + (1 - F_X(y_i)) \mathbb 1 (z_i = y_i) \right) \\ &= \prod_{i=1}^n \left(\lambda e^{-\lambda z_i} \mathbb 1 (z_i \ne y_i) + e^{-\lambda y_i} \mathbb 1 (z_i = y_i) \right) \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)} \prod_{i=1}^n e^{-\lambda z_i} \\ &= \lambda^{\sum_{i=1}^n \mathbb 1(z_i \ne y_1)} e^{-\lambda n \bar z}. \end{align*}$$ Notice here that the density and survival functions we choose are for $X$, not on $Y$ or $Z$! Then the log-likelihood is $$\ell (\lambda \mid \boldsymbol z, \boldsymbol y) = ( \log \lambda ) \sum_{i=1}^n \mathbb 1 (z_i \ne y_i) - \lambda n \bar z,$$ and we solve for the extremum as usual, giving $$\hat \lambda = \frac{\sum_{i=1}^n \mathbb 1(z_i \ne y_i)}{n \bar z},$$ where the numerator counts the number of paired observations that are not equal, and the denominator is the sample total of $z$.

Simulation of this is straightforward and I invite you to try it out to confirm the estimator works. Here is code in Mathematica to perform the estimation based on a sample of size $n$ and any $\lambda = t$:

F[n_, t_] := RandomVariate[TransformedDistribution[{Min[x, y], y},
             {Distributed[x, ExponentialDistribution[t]], 
             Distributed[y, ExponentialDistribution[1]]}], n]

T[d_] := Length[Select[d, #[[1]] != #[[2]] &]]/Total[First /@ d]

T[F[10^6, Pi]]

The last expression evaluates $\hat \lambda$ for $n = 10^6$ and $\lambda = \pi$. I got $3.14452$ when I ran it.