I have to show that the following set: $$A = \left\{ 1, 1-x , 1-2x + {1 \over 2} x^2\right\}$$
is orthogonal system in relative to the inner product
$$\langle f, g\rangle = \int ^\infty_0 f(x)g(x)e^{-x}dx$$.
as far as I know, in order that $A$ will be a orthogonal system, for any $f,g \in A$,
$$\langle f, g\rangle = \begin{cases}0 & f \ne g \\ 1 & f = g \end{cases}$$
Lets take $f = 1, g = (1 - 2x + \frac{1}{2}x^2)$,
\begin{align} \langle f, g\rangle = &\int^\infty_0 1\cdot(1 - 2x + \frac{x^2}{2})dx\\ = & \int^\infty_01-2x + \frac{x^2}{2} dx \\ = & \int^\infty_01 - 2x + \frac{1}{2}x^2 \\ = & \int^\infty_01 - \int^\infty_02x + \int^\infty_0 \frac {1}{2} x^2\\ = & x - \frac{x^2}{2} + \frac{x^3}{6}\bigg|^\infty_0 \\ = & 0 - (\infty - \frac{\infty^2}{2} + \frac{\infty^3}{6}) = \infty. \end{align}
So how that set is orthogonal? thanks in advance.
You've forgotten the $e^{-x}$ term in $\langle f,g\rangle$.
$$\langle f,g\rangle =\int_0^\infty 1\cdot(1-2x+\frac{x^2}{2})\cdot e^{-x} \mathrm{d}x=0$$
etc.