I am doing fits to some data and currently using mathematicas nonlinearmodel fit to generate fits and CIs, as well as doing bootstrapping for CIs for completeness. However, I would like to know how to generate CIs without bootstrapping (basically so i can understand what MM is doing and to check its values)
My rather poor understanding is as follows:
For a function of a single parameter $$f(a)$$ the likelihood function is the product of the probabilities
$$ P(X|f(a)) = \prod_{i}^N P(x_i|f(a)) $$ which if the residuals are normally distributed gives the likelihood function as an exponential $$ L =\frac{ \exp[-SSR/2\sigma^2]}{(\sigma \sqrt{2\pi})^N} $$ which one can then maximimize to get the best fit parameters, and can look for 95% CIs between $$\pm 1.96 \sigma$$
The first part of my question is: is it also reasonable to generalize the method for finding the CI boundaries by looking for the bounds of integration such that $$ \int_{a_0-\delta a}^{a_0+\delta a} L da = 0.95 $$ where a_0 is the best fit value.
If so, does this immediately generalize to the case of two parameters? I.e., define the CI contours by the double integral of L, or does this lead to a uniqueness issue? (thinking about integrating surfaces makes my head hurt a bit)
From what i've read this is usually deal with via a Chi squared/multinormal distribution after taking a series expansion of the log likelihood near the best fit parameter vector. Presumably, one then does a similar integral implicitly when looking up the appropriate quantiles of the resulting distribution, which is no doubt faster/neater, but in principle is the above approach correct or am i yelping up the wrong tree completely?
Thanks in advance
What you are describing is something like a "Wald confidence region", where you approximate the multivariate log-likelihood by a quadratic form. The preferred method is to use the chi-square approximation to the distribution of the likelihood ratio. For multiparameter estimation, you will need to adjust the degrees of freedom on the approximating chi-square distribution based on the number of parameters and then select the approparate likeliood ratio cutoff to get your approximate confidence region. This region will be a likelihood contour, not an ellipse as with the Wald confidence region.
This paper will explain in more detail how to get confience regions for MLEs