Help with understanding point from Kahneman's book “Thinking Fast and Slow”

Question

My question: How did Kahneman arrive at the 60% number in the last sentence ("60% of the pairs")?

From Daniel Kahneman, Thinking Fast and Slow (Chapter 19, Illusion of Understanding):

Update: from your answers, it appears that the number should have actually been 65%, so it was wrong in the book.

---

A very generous estimate of the correlation between the success of the firm and the quality of its CEO might be as high as .30, indicating 30% overlap. To appreciate the significance of this number, consider the following question:

Suppose you consider many pairs of firms. The two firms in each pair are generally similar, but the CEO of one of them is better than the other. How often will you find that the firm with the stronger CEO is the more successful of the two?

In a well-ordered and predictable world, the correlation would be perfect, and the stronger CEO would be found to lead the more successful firm in 100% of the pairs. If the relative success of similar firms was determined entirely by factors that the CEO does not control (call them luck, if you wish), you would find the more successful firm led by the weaker CEO 50% of the time. A correlation of .30 implies that you would find the stronger CEO leading the stronger firm in about 60% of the pairs—an improvement of a mere 10 percentage points over random guessing, hardly grist for the hero worship of CEOs we so often witness.

Answer 1

If you have two random variables $X,Y$, the correlation coefficient is

$$\rho=\frac{\mu_{XY}-\mu_X \mu_Y}{\sigma_X \sigma_Y}$$

where $\mu$ denotes the mean of the variable and $\sigma$ denotes the standard deviation of the variable. Suppose now you have two Bernoulli random variables $X,Y$ (so they take on only the values $0$ and $1$) and both have total probability $0.5$ of being $1$. In this case the correlation coefficient $\rho$ is

$$\frac{P(X=1,Y=1)-\frac{1}{4}}{\frac{1}{4}}=4P(X=1,Y=1)-1.$$

So $P(X=1,Y=1)=\frac{\rho+1}{4}$. The more interesting quantity is

$$P(X=1 \mid Y=1)=\frac{P(X=1,Y=1)}{P(Y=1)}=\frac{\rho+1}{2}.$$

Thus, if the correlation coefficient between CEO quality and firm success is $\rho$ and the randomly chosen CEO is better, then the probability that the firm is successful is increased by $\frac{\rho}{2}$, relative to a firm which is equally likely to have a good CEO vs. a bad CEO. So in this version of the model, the number should actually have been $65\%$.

Answer 2

For (mostly my own) future reference, here's my intuitive explanation of the answer.

If CEO strength made no difference, then for the stronger CEO, for 100 firms, 50 would be 'more successful' and 50 would be 'less successful'.

If a stronger CEO made a difference (.30 correlation), then out of 100 firms, 30 would be 'more successful' for the stronger CEO because of the .30 correlation. For the remaining 70, the 'more successful' firms would be distributed equally between the stronger and weaker CEOs (35 + 35).

So, the stronger CEO gets 30 + 35 = 65 'more successful' firms. Hence 65%.

Answer 3

Here another intuitive explanation, slightly different than the other answers.

We agree that a correlation of 0 means pure randomness, i.e. a 50% probability and a correlation of 1 means pure determinism, i.e. a 100% probability.

We now linearly interpolate and get the following linear relationship between probability and correlation:

probability = 50% + correlation * 50%

Observe that it satisfies (0, 50%) and (1, 100%). For a correlation of 0.3 the formula gives us a probability of 65%.

The answer 60% provided in the book appears to be wrong, or more details on the assumptions need to be provided. Note that also the value of 65% assumes a linear relationship between correlation and probability. I honestly do not know if this assumption can always be made.

Answer 4

I wrote to Kahneman to ask him about this, and he answered:

"I asked a statistician to compute the percentage of pairs of cases in which the individual who is higher on X is also higher on Y, as a function of the correlation. 60% is the result for .30. ... He used a program to simulate draws from a bivariate normal distribution with a specified correlation."

Source: Private email conversation with Daniel Kahneman on 2020-10-20

So we apparently cannot use the simple linear equation from chapter 18 for this.

This answers the question of how Kahneman arrived at the number 60%, in his own words. And it shows that all other answers in this thread are wrong: the correct answer is in fact not 65% as everyone else agrees. The question did not ask for detailed instructions on how to replicate this result.