Question about Logistic Regression - 4

Question

I am currently studying on logistic regression. So I have found a document on the Internet explaining about it.

Somehow, it explains Bernoulli distribution in the beginning and I am having a problem to understand the equation below. I can refer to the variance explanation in wikipedia, it is still not clear for me to understand. I am new to this, and reading the paper line by line. If you think you can give me an explanation, I will be really grateful.

I hope I have provided enough information for you to give me an explanation.

Thank you.

Answer 1

For the expected value, sum the values times the probabilities of the values: $$ \begin{align} \mathrm{E}[Y_i] &=\mathrm{P}[Y_i=0]\cdot0+\mathrm{P}[Y_i=1]\cdot1\\ &=(1-\pi_i)\cdot0+\pi_i\cdot1\\ &=\pi_i \end{align} $$ The variance is the mean of the squares minus the square of the mean. The mean of the squares is $$ \begin{align} \mathrm{E}[Y_i^2] &=\mathrm{P}[Y_i=0]\cdot0^2+\mathrm{P}[Y_i=1]\cdot1^2\\ &=(1-\pi_i)\cdot0+\pi_i\cdot1\\ &=\pi_i \end{align} $$ Therefore, $$ \begin{align} \mathrm{Var}[Y_i] &=\mathrm{E}[Y_i^2]-\mathrm{E}[Y_i]^2\\ &=\pi_i-\pi_i^2\\ &=\pi_i(1-\pi_i) \end{align} $$

Answer 2

By definition $Y_i$ is random variable which assumes the values $\{0,1\}$. So, for all $i$, $y_i\in\{0,1\}$. Then $P(Y_i = 1):=\pi_i$ and $P(Y_i = 0)= 1-P(Y_i = 1)= 1-\pi_i.$ You can summarize these information in

$$P(Y_i = y_i):=\pi^{y_i}_i(1-\pi_i)^{1-y_i}.$$

This notation is not precise as, in the context of logistic regression, one should consider conditional probabilities $$P(Y_i = 1)| X = x_i):=\pi(x_i,\beta)$$

instead, where $\pi(x_i,\beta)$ is defined using the inverse of the logit function. Explicitly $$\pi_i(x_i,\beta)=\frac{1}{1+e^{-\langle \beta, x_i\rangle}}.$$ In summary

$$E(Y_i)=\sum_{j=0}^1 y_j P(Y_i = j) = 1\cdot P(Y_i = 1) + 0\cdot P(Y_i = 0) =\pi_i. $$

Similar considerations hold for the variance.