Let $I_G\left[a,\:b\right]$ be the value of the Gaussian Quadrature from $a$ to $b$.
There are $n$ $+$ $1$ parameters. Prove that for all $f(x)$ with degree less than or equal to $2n$ $+$ $1$, $I_G\left[a,\:b\right]=\int _a^b\:f\left(x\right)dx$.
I understand that in this case, $I_G\left[a,\:b\right]=\sum _{i=0}^n\:A_ig\left(t_i\right)$, and that the value is supposed to be an approximation.
But I am quite confused on how to show that with the degree less than or equal to $2n$ $+$ $1$, then the value actually becomes fully accurate.
Any help would be greatly appreciated!
Note, the literature usually states it as $2n-1$, this causes just a relabeling of what you mean by "first order", "second order". The canonical definition is more intuitive, since second order refers to $n=2$, instead of $n=1$ in your case. In any event, this does not affect the answer.
The reason is that you use the abscissa as the roots of the $(n+1)^\textrm{st}$ order special polynomial. There are $n+1$ equations that solve for the $n+1$ weights that guarantee integrating any polynomial of degree at most $n+1$. You actually haven't said anything about the abscissa at this stage, so you are free to choose them. The great insight is to use the roots of the $n+1$ special polynomial associated with the problem at hand. It looks like you would use Legendre in the formulation of your question.
We choose this because any polynomial of order $2n+1$ can be polynomial divided by the $n+1$ special polynomial to get a quotient $q(x)$ and remainder $r(x)$ that are both at most degree $n$ $$ f(x) = P_{n+1}(x)q(x) + r(x). $$ When integrated using Gaussian quadrature $$ I[f] = \sum_{i=0}^n w_i P_{n+1}(x_i)q(x_i) + \sum_{i=0}^n w_ir(x_i) $$ the first term vanishes by the choice of abscissa. The remainder term is exactly integrated by the technique by the choice of weights. This is how the "extra" orders of accuracy come from.