Consider the vector space $P_3(\mathbb{R})$ of real polynomials of degree ≤2.
Let $\lambda\in \mathbb{C}\backslash\mathbb{R}$ be a complex number, which is not a real number.
Now let $p(\lambda)\overline{q(\lambda)}+p(1)q(1)+\overline{p(\lambda)}q(\lambda)$ be a real number for all $p,q \in P_3(\mathbb{R})$.
1) Show that $\langle p,q\rangle=p(\lambda)\overline{q(\lambda)}+p(1)q(1)+\overline{p(\lambda)}q(\lambda)$ defines an inner product on the real vector space $P_3(\mathbb{R})$.
I'm not certain if I've understood it correctly. But is it simply showing that it satisfies the definition of an inner product? That definition I have is from wolfram alpha:
$$1) \langle u+v,w \rangle=\langle u,w\rangle+\langle v,w\rangle$$ $$2) \langle\alpha v, w\rangle=\alpha \langle v,w\rangle$$ $$3) \langle v,w\rangle =\langle w,v\rangle$$ $$4) \langle v,v\rangle\geq 0$$
However after trying for a bit I can't seem to figure it out.
You indeed have to verify each of the assertions:
(1),(2) and (3) are quite easy using properties of addition and multiplication over $\mathbb{C}$.
(4): $\langle p,p\rangle = 2|p(\lambda)|^2+p(1)^2\geqslant 0$
But that is not all, you must also verify that $\langle p,p\rangle = 0 \Rightarrow p=0$. Which comes from the fact that $2|p(\lambda)|^2+p(1)^2 = 0 \Rightarrow p(\lambda) = 0$ and $p(1)=0$. However, $\lambda \neq 1$ and $\overline{\lambda}$ ($\neq \lambda$ since $\lambda$ is not real) is also a root (because $p$ is a real polynomial) which makes 3 roots for a polynomial of degree 2 meaning $p=0$.
As Michael Hoppe noticed, you also have to check that $\langle p,q\rangle \in \mathbb{R}$ for $p,q\in P_3(\mathbb{R})^2$. That is true because $p(1)q(1)$ is real and $p(\lambda)\overline{q(\lambda)}+\overline{p(\lambda)}q(\lambda)=p(\lambda)\overline{q(\lambda)}+\overline{p(\lambda)\overline{q(\lambda)}}=2\text{Re}(p(\lambda)\overline{q(\lambda)})\in \mathbb{R}$ where $\text{Re}$ denotes the real part of a complex number.