Jaynes' derivation of the laws of probability

408 Views Asked by At

I have asked about parts of this before, and now that I understood that explanation, a new doubt showed up in the remaining parts of the derivation.

We have the functional equations $xS(\frac{S(y)}{x}) = yS(\frac{S(x)}{y})$ and $S[S(x)] = x$, the definitions $\frac{S(x)}{y} = 1-e^{-q}$, $S(1-e^{-q}) = e^{-J(q)}$ and $\alpha \equiv \log(\frac{-xS'(x)}{S(x)})$ and the theorem $\frac{S(y)}{x} = 1-e^{-(\alpha+q)}+O(exp\{-2q\})$. From these, Jaynes transforms the functional equation into $J(q+\alpha)-J(q) = log(\frac{x}{S(x)})+log(1-exp\{-q\})+O(exp\{-2q\})$.

However, for that to be true, we'd need that $S[1-e^{-(\alpha+q)}+O(exp\{-2q\})] = S[1-e^{-(\alpha+q)}]exp\{O(exp\{-2q\})\}$ which isn't justified, and doesn't even sound all that plausible.

Furthermore, from that he jumps to $J(q) \approx a + bq + O(exp\{-q\})$ where $b=\alpha^{-1}log(\frac{x}{S(x)})$.

Can anyone explain to me how he made these two jumps?

--EDIT:

I understand the last jump now, I think, and someone said I should use Taylor's theorem on S around 1 to get $S(1-e^{-(\alpha+q)})+O(exp\{-2q\})$ from $S[1-e^{-(\alpha+q)}+O(exp\{-2q\})]$ and then use Taylor's theorem again to jump from $\log[S(1-e^{-(\alpha+q)})+O(exp\{-2q\})] = \log[e^{-J(q+\alpha)}+O(exp\{-2q\})]$ to $\log[e^{-J(q+\alpha)}]+O(exp\{-2q\}) = -J(q+\alpha)+O(exp\{-2q\})$.

I don't quite see it either. Does anyone know how this is done?

1

There are 1 best solutions below

0
On BEST ANSWER

Okay so apparently the idea is using the definition of a differential $f(x+d)\approx f(x)+f'(x)d$ for sufficiently small $d$. Then we take $S[1 - e^{-(\alpha+q)}+O(e^{-2q})] \approx S[1 - e^{-(\alpha+q)}]+S'[1 - e^{-(\alpha+q)}]O(e^{-2q})$.

After that, we see that $$\log(S[1 - e^{-(\alpha+q)}+O(e^{-2q})]) \approx \log[S[1 - e^{-(\alpha+q)}]+S'[1 - e^{-(\alpha+q)}]O(e^{-2q})] \approx \\ \log[S(1 - e^{-(\alpha+q)})] + \frac{(S'(1 - e^{-(\alpha+q)}))^2O(e^{-2q})}{S(1 - e^{-(\alpha+q)})}$$

Then, replacing our definitions in the original functional equation $xS(\frac{S(y)}{x}) = yS(\frac{S(x)}{y})$ we get $xe^{-J(q)} = yS[1 - e^{-(\alpha+q)}+O(e^{-2q})]$ and if we divide both sides by S(x) and take the log we get to

$$J(q+\alpha) - J(q) = \log(\frac x {S(x)})+\log(1-e^{-q})+\frac{(S'(1 - e^{-(\alpha+q)}))^2O(e^{-2q})}{S(1 - e^{-(\alpha+q)})}$$

The last two terms approach $0$ as $q\rightarrow +\infty$ and since $J(q)$ is independent of $x$ it must be true that asymptotically it is linear, $J(q) \approx a + bq + O(e^{-q})$ where $b=\alpha^{-1}\log(\frac{x}{S(x)})$ is some constant.

Jaynes then uses that last fact to eventually find that $S(x) = (1-x^m)^{1/m}$ where $0 < m < +\infty$ is some constant, and shows that this function is necessary and sufficient for his purposes and thus we can take $m$ to have any value in the permitted range.

At least, that was the solution I could find.