Consider the following quadrature:
$$\int_0^1 x^cf(x)dx\approx Af(0)+B\int_0^1 f(x)dx, c>1, \neq 0$$
Determine A and B such that this rule has a degree of exactness 1.
Let $E(f)$ be the error functional of the determined rule.
Show that the Peano Kernel $K_1(t)\geq 0$ if $c>0$ and $K_1(t)\leq 0$ if $c<0$
I determined $A=\dfrac{-c}{c^2+3c+2}$ and $B=\dfrac{2}{c+2}$ but I cannot derive an expression for the error.
The constants $A,B$ are well determined. Assuming that our quadrature is exact on affine functions, the Peano Kernel of 1st order should be computed. This is $K_1(x)=E\bigl((\cdot - x)_+\bigr)$. Did you try to compute it? We have $$K_1(x)=\int_x^1 t^c(t-x)\,\text{d}t-\frac{(x-1)^2}{c+2}.$$ Two critical cases are $c=-1$ or $c=-2.$ They should be analysed separately. To compute the error, use Peano Kernel Theorem. If you know that $K_1$ has no sign changes, the only thing you should do is to compute $$\int_0^1 K_1(x)\,\text{d}x,$$ which is the factor of $f''(\xi),$ i.e. the error of this quadrature is $$f''(\xi)\int_0^1K_1(x)\,\text{d}x$$ for some $\xi\in[0,1].$