Differentiating the posterior distribution function

Question

I am learning about Bayesian statistics and I'm currently doing loss functions. Let $f(\theta | \mathbf{x} ) $ be a posterior pdf . Let $F(\theta | \mathbf{x} ) $ be the associated distribution function. I want to differentiate $F(a - D| \mathbf{x} )$ with respect to $D$. In this case $D$ is the "decision" which is to be optimised and $a$ is constant. I am having a problem here: $$\frac{d}{d D}F(a - D| \mathbf{x} ) = \frac{d}{d D} \left( \int_{-\infty}^{a - D} f(\theta|\mathbf{x}) d \theta \right)$$ $$=f(a - D | \mathbf{x})$$

Is this correct ? I think it might be wrong. Should it be $=-f(a - D | \mathbf{x})$ because I have to apply the chain rule somewhere ?? Or is something else wrong ? I know there are some issues about integration under differentiation but my teacher said I don't need to worry about that now, and just use this: $$\frac{d}{dy} \int_{-\infty}^y f(t) dt=f(y)$$

I'm doing self-study (with a bit of teacher guidance in his own time so I don't like to ask him too much) but I feel a bit out of depth now and school breaks up for the holidays next week !

Answer 1

Think of this as a special case of differentiating $h(f(D))$ with respect to D:

$h(f(D)) = \int_{-\infty}^{f(D)}g(x)\text{d}x$

where $\text{d}h/\text{d}D = h'(f(D))f'(D)$ via the chain rule, as you thought. In your case, $f(D) = a-D$, so $h'(f(D)) = g(a-D)$, as you know, and $f'(D) = -1$.

Answer 2

I think the key here is to abstract away the irrelevant information. That might be easier said than done, but consider this line of reasoning:

• The question is about a derivative.

• It's not about numerical calculation of the derivative; symbolically, we know that $dF = f$. (Observe how the dependence on $\mathbf{x}$ is dropped in this simplified notation: because $\mathbf{x}$ is fixed throughout the problem, it likely plays no essential role.)

• The difficulty appears to stem from differentiating with respect to $D$ when the argument of $F$ is supplied as $a-D$ instead of $\theta$.

• Although there is a context or "story line" in which decisions, losses, and a Bayesian procedure are involved, this context appears to offer no additional mathematical information.

A propos the last point, one could hope that a solution might be motivated by the context, but ultimately a rigorous mathematical answer should not need to invoke any element of this context. This makes it a good candidate to abstract away.

Whence, let's try reframing the question:

Given that the derivative of $F$ is $f$, to produce an expression for $\frac{d}{dD}F(a-D)$.

Many people visualize this situation as a composition of functions, written thus:

$$D \to a - D \to F(a-D)$$

The first function sends $D$ to $a-D$; the second is $F$.

The Chain Rule states that when everything makes sense, the composition has a derivative and the derivative is the product of the derivatives of the arrows. The derivative of $D \to a-D$ is $-1$ (an elementary calculation) while the derivative of $F$ is (symbolically) $f$, applied (as in the diagram) to $a-D$, whence the derivative of this composition is simply $-1 \times f(a-D)$. Reverting to the original notation, we would write

$$\frac{d}{dD} F(a-D|\mathbf{x}) = -f(a-D|\mathbf{x}).$$

---

Please note that the potentially confusing question of differentiating a limit of an integral doesn't even come up: that's one advantage of the clarity and focused afforded by this method of abstracting out the inessential parts of a "real world" problem.