We need to prove that for every real polynomial $f$, $$\bigg| \int_{-1}^{1}(|x|^3 f(x) + 6xf'(x))\,\mathrm{d}x \bigg| \leq \frac{5}{\sqrt{3}}\bigg|\int_{-1}^{1}(|x||f(x)|^2 +3|f'(x)|^2) \,\mathrm{d}x \bigg|^\frac{1}{2}.$$
I assume this inequality has been reached by an application of Cauchy-Schwarz on the vector space of continuous functions from $[-1,1]$ to $\mathbb{R}$, but how can we find the scalar product it uses? And how can we determine which functions in that space are being multiplied to get the integral on the left? (It seems that one of them has norm $\frac{5}{\sqrt{3}}$, but it's hard to see how that helps us.) Does this inequality arise from a standard product on that space that I just haven't seen?
One choice would be $\displaystyle \langle u, v \rangle = \int_{-1}^1(|x|uv+3u'v')dx$. Check the axioms, and then consider the CS inequality for vectors $x^2$ and $f$.
To finish, note $\displaystyle \|x^2 \|^2 = \int_{-1}^1(|x|^5+12x^2)dx = \frac{25}3$.
Looking backward, can you see why this choice works?
If we had the choice of vector space as well, this may have been easier considering a slightly different space, composed of couples of form $(f_1, f_2)$ where $f_1, f_2$ are real valued continuous functions, ie $C^2([-1, 1] \mapsto \mathbb R)$. Equip it with the simpler inner product $$\langle (f_1, f_2), (g_1, g_2)\rangle = \int_{-1}^1(f_1g_1+f_2g_2)dx$$ You will need to verify the relevant axioms.
Now consider the CS inequality $$\left\langle\left(|x|^{5/2}, \sqrt{12}x\right), \left(|x|^{1/2}f, \sqrt3f'\right)\right\rangle^2 \leqslant \left\| \left(|x|^{5/2}, \sqrt{12}x\right)\right\|^2 \times \left\|\left(|x|^{1/2}f, \sqrt3f'\right) \right\|^2$$ $$\implies \bigg| \int_{-1}^1\left(|x|^3 f + 6xf'\right)\,dx \bigg|^2 \leqslant \color{red}{\int_{-1}^1\left(|x|^5 +12x^2\right) \,dx} \cdot \int_{-1}^1\left(|x|\,f^2 +3f'\,^2\right) \,dx $$
which is the same result we need. Looking backwards, do you see how the choice in red and hence the choice of vectors comes about?