Taylor expansion in Logistic regression

Question

More of a question about notation of the Taylor expansion in Logistic regression that anything else.

Given a nonlinear cost function $ J(\theta)= - \frac{1}{n} \sum_{i=1}^n y_i (b+x_iw) - ln(1+e^{(b+x_iw)}) $ and given a Taylor expansion to approximate the cost function $ f(\theta) \approx f(\theta^{(t)})+f^{\prime}(\theta^{(t)}) (\theta^{(t+1)}-\theta^{(t)}) $. What I want to know is if the constant term is supposed to be a derivative of the cost function $ f(\theta) = \frac{d}{d\theta} J(\theta) = J^{\prime}(\theta) $ or is the constant term just equal to the cost function $ f(\theta) =J(\theta) $

I found differing information here and here

Answer 1

In general for Taylor expansion series, the constant term should be close to the value you are trying to approximate.

Therefore, if you are trying to approximate $J(\theta^{(t+1)})$, then the first term of your expansion should take a known value of $J$, such as the $J(\theta^{(t)})$.

If you are trying to approximate $J'(\theta^{(t+1)})$, then the first term should be $J'(\theta^{(t)})$.

Answer 2

Found the answer here

Depending on the context, Newton's method can mean one of the two below:

• In calculus, Newton's method for root finding is a root-finding algorithm which produces successively better approximations to the roots (or zeroes) of a real-valued function.

• In optimization, Newton's method for optimisation is applied to the derivative f ′ of a twice-differentiable function f to find the roots of the derivative (solutions to f ′ (x) = 0), also known as the stationary points of f. These solutions may be minima, maxima, or saddle points.