From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\mu \cdot x- \lambda A(\eta))$ and the posterior is $\pi^*(\eta|x) = \pi(\eta | \mu + T(x), \lambda + 1)$ [1].
I am confused by my bayesian professor (in a undergrad class) with respect of findind these parameters, $\mu$ and $\lambda$.
My question is: when we start with "standard" parameterization $f(x|\theta) \propto \exp(B(\theta) \cdot T(x) - A(\theta))$ and then reparameterize to natural parameter $\eta$ as above with $\eta = B(\theta)$, does $\pi(\theta | \mu, \lambda) = |B'(\theta)| \pi^*(B(\theta) | \mu, \lambda)$? Or does $\pi(\theta | \mu, \lambda) \propto \pi^*(B(\theta) | \mu, \lambda)$?
In other words, we can only substitute $B(\theta)$ on $\eta$? Or do we need to consider the jacobian, since $B(\theta)$ is a transformation of the random variable $\theta$?
If only substitution is enough, then we conclude that jacobian does not change the family of $\pi$. And in that case, the "correct" hyperparameters $\mu$ and $\lambda$ are not of importance to bayesian inference. But in that case, the table from the reference book [1] does not hold.
.
For example, if $x \sim Poisson(\theta)$ then $f(x | \theta) \propto \exp\{B(\theta)x - \theta\}$, where $B(\theta) = \ln(\theta)$. Hence $\eta = \ln(\theta)$ and $f(x | \eta) \propto \exp\{\eta x - e^{\eta}\}$. Therefore $\pi^*(\eta | \mu, \lambda) \propto \exp\{\mu \eta - \lambda e^{\eta}\}$.
Now we have $|B'(\theta)| = \theta^{-1}$ and $\pi(\theta | \mu, \lambda) \propto \theta^{\mu-1} e^{-\lambda \theta}$, which is Gamma$(\mu, \lambda)$. If we do not consider the jacobian, $\pi(\theta | \mu, \lambda) \propto \theta^{\mu} e^{-\lambda \theta}$, which is Gamma$(\mu+1, \lambda)$.
So there is a correct parameterization? Notice that both of them follow the posterior of this table (under a reparameterization to $\alpha$ and $\beta$).
[1] Robert, Christian. The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer Science & Business Media, 2007.