Logit regression change in $x$

42 Views Asked by At

I've been reviewing the logit regression equation to predict the probability of a response variable given $X=x$ as per below and I can't understand how a per unit change in the predictor is equal to $e^B$ change in the probability:

$$p(x) = e^{Bx}/(1+e^{Bx}) \qquad \text{Eq.(2)}$$ where $x$ is the predictor variable.

Rearranging Eq.(2) to obtain $$e^{Bx} = p(x)/(1-p(x)) = y$$

such that $y = e^{Bx} \implies dy = By \implies dy = Be^{Bx}$ where $dx = 1$ and we differentiate by $x$.

However if I natural $\log()$ Eq.(2) $$\implies Bx = \ln(p(x)/(1-p(x))) \\ \implies Bdx = \text{delta of the}~ \ln(p(x)/(1-p(x))) \\ \implies \ln(y_1) = B + \ln(y_0)$$ by discretization which if you raise both sides by $e$ then you get correctly $y_1 = e^B y_0$.

However from a derivative perspective I cannot seem to arrive at $e^B$ given $dy = Be^{Bx}$?

1

There are 1 best solutions below

0
On

The assertion is that a unit increase in the predictor $x$ results in a change in the odds by a factor of $e^B$. In other words, if $f(x):=p(x)/[1-p(x)]$ denotes the odds at $x$, the assertion is: $$ f(x_0+1) = e^B f(x_0).\tag1 $$ Since $f(x)=e^{Bx}$, as you've stated, equation (1) can be proved simply by plugging in $x_0$ and $x_0+1$ into the formula for $f$, and simplifying. There's no need to take derivatives.

If you try the derivative approach on $f(x)$, you will get a linear approximation of the form: $$ f(x_0+1)-f(x_0)\approx f'(x_0).\tag2 $$ Plugging in $f'(x)=Be^{Bx}=Bf(x)$, you can rearrange (2) into the form: $$ f(x_0+1)\approx f(x_0) + Bf(x_0)=(1+B)f(x_0). $$ This is the closest you'll get to (1). Remember this is only a linear approximation, which is OK when $B$ is small, since $e^B\approx 1+B$ in that case, but otherwise not so close. The reason why the derivative approach works better with $\ln f$ (as you've done) than with $f$ is that the result (1) is asserting a multiplicative change in $f$ when you increase $x_0$ by $1$; this means that you'll get a linear increase in $\ln f$ after a unit increase in $x_0$, as you can see by logging both sides of (1): $$ \ln f(x_0+1) - \ln f(x_0) = B.\tag3 $$ Because of the form of $f$, the linear approximation for $\ln f(x_0+1)-\ln f(x_0)$ will exactly get you equation (3). But again, you don't need calculus to prove (3) or (1).