Problem
Let $X \sim exp(\theta)$ and $X_1,..,X_n$ a random sample from $X$. I've computed the MLE and the Fisher information number : \begin{align*} \theta_{MLE} &= \frac{1}{\bar{X}} \\ I(\theta) &= \frac{1}{\theta^2} \end{align*} I therefore obtain the following asymptotic distribution for $\lambda_{MLE} $ : \begin{align} \sqrt{n} (\theta_{MLE} - \theta_0) \xrightarrow{D} \mathcal{N}(0,\theta^2) \end{align} However I now wish to derive an exact confidence interval for $\theta$.
Attempt of solution
I know that : \begin{align} \sum_{i=1}^n X_i &\sim Gamma(n,\theta) \\ \frac{1}{n}\sum_{i=1}^n X_i &\sim Gamma(n,\theta/n) \end{align} So an exact confidence interval for $\mu$ would be : \begin{align} P_{\mu}\Big\{F^{-1}(\alpha/2)_{n,\theta/n} \leq \mu \leq F^{-1}(1-\alpha/2)_{n,\theta/n}\Big\} = 1-\alpha \end{align} Where $F^{-1}(\alpha/2)_{n,\theta/n}$ denotes the $\alpha/2$ quantile of a Gamma distribution with shape parameter $n$ and scale parameter $\theta/n$. So knowing that if $X \sim Exp(\theta)$ : \begin{align} \theta = 1/\mu \end{align} I derive an exact confidence interval for the parameter $\theta$ of an exponential distribution : \begin{align} P_{\theta}\Bigg\{\Big[F^{-1}(1-\alpha/2)_{n,\theta/n}\Big]^{-1} \leq \theta \leq \Big[F^{-1}(\alpha/2)_{n,\theta/n}\Big]^{-1} \Bigg\} = 1-\alpha \end{align} Is that correct ?
Yes, your work is correct for an exact, equal-tailed confidence interval; that is, the probability that the interval exceeds the true parameter equals $\alpha/2$, which is also the probability the interval is below the true parameter. One could construct an interval that does not allocate the total error $\alpha$ equally, but still has the same coverage probability $1 - \alpha$.
An alternate formulation of the lower and upper confidence limit can be obtained by noting that the distribution of $1/\bar X$ is inverse gamma with PDF $$f_{\theta_{\text{MLE}}} (s) = \frac{(n \theta/s)^n e^{-n \theta/s}}{s \Gamma(n)}, \quad s > 0.$$ Thus the lower and upper limits may be expressed directly in terms of quantiles of this distribution, namely $$\Pr\left[F_{\theta_{\text{MLE}}}^{-1}(\alpha/2) \le \theta \le F_{\theta_{\text{MLE}}}^{-1}(1 - \alpha/2)\right] = 1 - \alpha.$$