Question
Let $X \mid \mu \sim \mathrm{Poisson} (\mu)$ and $\mu \sim \mathrm{Gamma} (1, 1)$ and suppose that a very large number $x$ is observed. Find, in terms of $x$, an approximate $95\%$ Bayesian credible interval for $\mu$. (Hint: $\mathrm{Gamma} (n, 1)$ is the distribution of the sum of $n$ independent $\mathrm{Exponential} (1)$ random variables).
My thoughts
The comments have pointed out some of my errors previously and following their suggestions, I have re-worked this.
I have found that $\mu \mid X \sim \mathrm{Gamma} (X + 1, \frac 1 2)$ using shape and scale parameters and since $x$ is large, the Central Limit Theorem can be applied, but I am unsure if the following is correct - does this mean $\frac 1 2 \mu \mid x \sim \mathcal{N} (x + 1, x + 1)$ approximately and since approximately $95\%$ of all normal data falls within $2$ standard deviations of the mean, then an approximate $95\%$ credible interval for $\mu$ is $[2(x + 1 - 2\sqrt{x + 1}), 2(x + 1 + 2\sqrt{x + 1})]$?
P.S. We have just covered Bayesian statistics and this is my first ever encounter of a problem asking for a credible interval, so any intuitive explanations will be greatly provided!
I'll write $\widetilde{\mu}$ to denote the random variable and $\mu$ to denote particular values. We're given the following probability density functions: $$\begin{align} f_{X\mid \widetilde{\mu}}(x\mid \mu)&=e^{-\mu}{\mu^x\over x!}{\bf 1}_{x\in\{0,1,2,...\}}\\[2ex] f_{\widetilde{\mu}}(\mu)&=e^{-\mu}{\bf 1}_{\mu>0} \end{align}$$ Therefore the posterior density function for $\widetilde{\mu}$ given $X=x\in\{0,1,2,,...\}$ is as follows, where we can write "$\propto$" ("proportional to") and leave out everything that doesn't depend on the distribution variable $\mu$: $$\begin{align} f_{\widetilde{\mu}\mid X}(\mu\mid x)&\propto f_{X\mid \widetilde{\mu}}(x\mid \mu)\,f_{\widetilde{\mu}}(\mu)\\[2ex] &\propto \mu^{(x+1)-1}e^{-2\mu}{\bf 1}_{\mu>0}\\[2ex] \end{align}$$ i.e., $(\widetilde{\mu}\mid X=x)\sim\text{Gamma(shape=$x+1$, rate=$2$)}.$
But to use the hint, we need to relate this to a Gamma distribution whose rate parameter is $1$, so we recall that for any random variable $Y$, the relation between the densities of $Y$ and $cY$ ($c$ a positive constant) is just $f_{cY}(t)\propto f_Y(t/c)$; so we have $$\begin{align} f_{2\widetilde{\mu}\mid X}(t\mid x)&\propto t^{(x+1)-1}e^{-t}{\bf 1}_{t>0}\\[2ex] \end{align}$$ That is, $(2\widetilde{\mu}\mid X=x)\sim\text{Gamma(shape=$x+1$, rate=$1$)}$. Now, according to the hint, this is the distribution of a sum of $x+1$ iid $\text{Exp($1$)}$ random variables, and we're given that $x$ is "very large"; consequently, we assume the Central Limit Theorem applies to give an approximation: $$ {(2\widetilde{\mu}\mid X=x)-E(2\widetilde{\mu}\mid X=x)\over \sqrt{V(2\widetilde{\mu}\mid X=x)}}\approx\mathcal{N}(0,1)\\[4ex] {(2\widetilde{\mu}\mid X=x)-(x+1)\over \sqrt{x+1}}\approx\mathcal{N}(0,1)\\[4ex] P\left(-z_{1-p/2}< {2\widetilde{\mu}-(x+1)\over \sqrt{x+1}}<z_{1-p/2}\ \middle|\ X=x\right)\approx 1-p\\[4ex] $$ where $z_q$ denotes the value such that $P(\mathcal{N}(0,1)<z_q)=q$.
An approximate $100(1-p)$% credible interval for $\widetilde{\mu}$, given $X=x$, is therefore an interval with endpoints $${1\over 2}(x+1) \pm {1\over 2}z_{1-p/2}\sqrt{x+1} $$ e.g., with $1-p=95$%:$${1\over 2}(x+1) \pm 0.98\sqrt{x+1}.$$