How would the sketch of a residual plot look for residuals from an exponential distribution with expectation 0?

Question

A linear model has been fitted under the usual assumptions, i.e. Y = Xβ + ε, with $ε ∼ N(0,σ^2I)$. How would the sketch of a residual plot look for residuals from an exponential distribution with expectation 0?

Answer 1

It is simpler to discuss residuals in a one-factor ANOVA, say with three levels $i = 1,2,3=g$ of the factor and $r=10$ replications per level. Suppose all $3n = 30$ observations are from $\mathsf{Exp}(\lambda = 1/5).$ The following data have no main effect $(\mu_1 = \mu_2 = \mu_3 = 5)$ because this exponential distribution has mean $5.$.

The residuals are of the form $X_{ij} - \bar X_{i\cdot}.$ A plot of residuals will tend to show upward skewness of the residuals in each group, with scattered high outliers. Similarly, residuals from a regression with exponential errors will tend to show concentrations of points below $0$ and scattered points (some outliers) above $0$, but these effects may be more difficult to recognize in a regression plot than in an ANOVA plot.

The top panel below is a plot of residuals by group. The boxplots in the bottom panel indicate that, for my data, four points are outliers within their groups. Residuals in each group average to $0.$

Answer 2

Here is a model with $\beta_0 = -8$ and $\beta_1$. As said @BruceET, in order to maintain expectation of $0$, I have to set $\lambda = 1/8$ in the exponential noise. In this case you can view $\epsilon_i \sim \mathcal{E}xp(\lambda) -\beta_0$, thus $\mathbb{E}{\epsilon_i}=0$. What you see on both plots is a random sample from exponential distribution, namely, large concentration at$[-8, 0]$ ( support left boundary up to the expected value) and then random "spikes" upwards.

set.seed(1)
x = rnorm(1000, 10, 2)

y = - 8 + x + rexp(1000, 1/8)

mod = lm( y ~ x)

par(mfrow=c(1,2))
plot(x, y, pch = 20,
     col      = "darkgrey",
     main     = "Linear Regression",
     cex.main = 0.9)

abline(mod)

plot(fitted(mod),
     residuals(mod),
     pch      = 20, 
     main     = "Residuals vs. Fitted values",
     cex.main = 0.9,
     col      = "darkgrey")
abline(h = 0)