There are two positive integers $m$ and $n$. Let $$f(x)=\int_1^x {(t-a)^{2n}(t-b)^{2m+1}}dt, a\neq b$$ I need to find whether $f(x)$ is maximum/minimum at $x=a,b$.
My Approach: I differentiated $f(x)$ with respect to $x$. $$f'(x)=\frac{d}{dx}{(\int_1^x {(t-a)^{2n}(t-b)^{2m+1}}dt)}$$ By Newton-Leibniz formula, $$f'(x)=\big[(x-1)^{2n}(x-b)^{2m+1}\big]\frac{dx}{dx}-\big[ (1-a)^{2n}(1-b)^{2m+1} \big]\frac{d}{dx}{(1)}$$ Thus, $f'(x)=(x-a)^{2n}(x-b)^{2m+1}$. For critical points, $f'(x)=0$, $$0=(x-a)^{2n}(x-b)^{2m+1}$$ So, $x=a$ and $x=b$ are critical points. $$f''(x)=(x-b)^{2m+1}\frac{d(x-a)^{2n}}{dx}+(x-a)^{2n}\frac{d(x-b)^{2m+1}}{dx}$$ $$f''(x)=\big[ (x-b)^{2m+1}(2n)(x-a)^{2n-1} \big]+\big[ (x-a)^{2n}(2m+1)(x-b)^{2m} \big]$$ From here one can easily deduce that $f''(x)\geq0$. Now, if I put the value of $x=a$ or $x=b$, I'll get $f''(x)=0$ which gives that the function $f(x)$ has inflexion at $x=a$ and $x=b$. So where am I going wrong because the answer given in my book is that $x=b$ is a point of local minimum.
$$f'(x)=(x-a)^{2n}(x-b)^{2m+1}$$ So let $\epsilon>0$, then $$f'(b+\epsilon)=(b-a+\epsilon)^{2n}(\epsilon)^{2m+1}>0.$$ Instead $$f'(b-\epsilon)=(b-a-\epsilon)^{2n}(-\epsilon)^{2m+1}<0.$$ So if you picture it, you'll see $x=b$ is a minimum.