Let $f: [0,\infty) \to [0,\infty)$ be a continuous function such that $f(f(x)) = x^2, \forall x \in [0,\infty)$. Prove that $\displaystyle{\int_{0}^{1}{(f(x))^2dx} \geq \frac{3}{13}}$.
All I know about this function is that $f$ is bijective, it is strictly increasing*, $f(0) = 0, f(1) = 1, f(x^2) = (f(x))^2, \forall x \in [0, \infty)$ and $f(x) \leq x, \forall x \in [0, 1]$**. With all these, I am not able to show that $\displaystyle{\int_{0}^{1}{(f(x))^2dx} \geq \frac{3}{13}}$.
*Suppose that $f$ is strictly decreasing. Then, $\forall x \in (0,1), x^2 < x \implies f(x^2) > f(x) \iff (f(x))^2 > f(x) \iff f(x) > 1$, which is false because, if we substitute $x$ with $0$ and with $1$ in $f(x^2) = (f(x))^2$ we get that $f(0) \in \{0,1\}$ and $f(1) \in \{0,1\}$. So $f$ is strictly increasing.
**Suppose that there exists $x_0 \in [0, 1]$ such that $f(x_0) > x_0$. Then, $x_0^2 = f(f(x_0)) > f(x_0) > x_0$, which is false. Then $f(x) \leq x, \forall x \in [0,1]$.
Edit:
I have come up with an idea to use Riemann sums, but I reach a point where I cannot continue.
Let $\epsilon < 1$. Then $f(\epsilon) = x_1$ and $f(x_1) = \epsilon^2$. And now $(f(\epsilon))^2 = f(\epsilon^2) = x_2$ and so on. Now we will use the Riemann sum:
We will take the partition $\Delta = (1 > \epsilon > \epsilon ^2 > ... \epsilon ^{2^n} >0 ) $and the intermediate points will be the left margin of each interval. Then we have:
$\displaystyle{\int_{0}^{1}{(f(x))^2 dx}} = \displaystyle{ \lim_{\epsilon \to 1}{\lim_{n \to \infty}{\sum_{k = 0}^{n}{(\epsilon^{2^k} - \epsilon^{2^{k+1}})\epsilon^{2^{k+1}}}}}}$
I do not know how to compute this.
It took a while, but I finally managed to have a solution.
I will show that $\displaystyle{\int_0^1{(f(x))^2dx} = 1-4\int_0^1{x^3f(x)dx}}$. (*)
For $n \in \mathbb{N}^*$, let $D_n = (\displaystyle{0, \frac{1}{n}, ..., \frac{n-1}{n}},1)$ and $E_n = (\displaystyle{0, f(\frac{1}{n}), ..., f(\frac{n-1}{n}),1})$ be two divisions on the interval $[0,1]$.
Evidently, $||D_n|| = \displaystyle{\frac{1}{n}}$, so $\displaystyle{\lim_{n \to \infty}{||D_n||} = 0}$.
Because $f$ is a continuous function, then it is also uniformly continuous, so $\displaystyle{\lim_{n \to \infty}{||E_n||} = 0}$.
Let $\xi_i = \displaystyle{\frac{i}{n}}$.
Now I will write the Riemann sum for $f^2$ for the division $E_n$(upper Darboux sum).
$S_{E_n}(f^2) = \displaystyle{\sum_{i=1}^n{(f(f(\xi_i)))^2(f(\xi_i) - (f(\xi_{i-1})))} = \sum_{i=1}^n{\xi_i^4(f(\xi_i)-f(\xi_{i-1}))}}$.
We can rewrite this sum:
$S_{E_n}(f^2) = \xi_n^4 f(\xi_n) - \xi_0^4 f(\xi_0) + \displaystyle{\sum_{i=1}^{n}{f(\xi_{i-1})(\xi_{i-1}^4 - \xi_i^4)} = 1- \sum_{i=1}^n{f(\xi_{i-1})(\xi_{i}^4 - \xi_{i-1}^4)} = 1 - \sum_{i=1}^n{f(\xi_{i-1})(\xi_i^3 + \xi_i^2\xi_{i-1} + \xi_i\xi_{i-1}^x + \xi_{i-1}^3)(\xi_i - \xi_{i-1})}}$.
Also, $4\xi_{i-1}^3 f(\xi_{i-1}) \leq f(\xi_{i-1})(\xi_i^3 + \xi_i^2\xi_{i-1} + \xi_i\xi_{i-1}^x + \xi_{i-1}^3)(\xi_i - \xi_{i-1}) \leq 4\xi_i^3 f(\xi_i)$.
Considering the function $h : [0,1] \to [0,1], h(x) = x^3f(x)$, we obtain that:
$s_{D_n}(h) < \displaystyle{\sum_{i=1}^n{f(\xi_{i-1})(\xi_i^3 + \xi_i^2\xi_{i-1} + \xi_i\xi_{i-1}^x + \xi_{i-1}^3)(\xi_i - \xi_{i-1})}} < S_{D_n}(h)$, where $s_{D_n}(h)$ and $S_{D_n}(h)$ represent the Darboux sums(lower and upper, respectively) for the function $h$ and the division $D_n$.
Therefore, we have that $\displaystyle{\lim_{n \to \infty}{\sum_{i=1}^n{f(\xi_{i-1})(\xi_i^3 + \xi_i^2\xi_{i-1} + \xi_i\xi_{i-1}^x + \xi_{i-1}^3)(\xi_i - \xi_{i-1})}} = \int_0^1{h(x)dx} = 4\int_0^1{x^3f(x)dx}}$.
Now, using the Cauchy-Schwarz inequality we have that:
$\displaystyle{\int_0^1{x^3f(x)dx} \leq \sqrt{\int_0^1{x^6dx} \cdot \int_0^1{(f(x))^2dx}} = \sqrt{\frac{\int_0^1{(f(x))^2dx}}{7}}}$. (**)
Let $a = \displaystyle{\int_0^1{(f(x))^2dx}}$.
Using (*) and (**), we obtain that $\displaystyle{0 < 1-a \leq 4\sqrt{\frac{a}{7}}}$, therefore $(1-a)^2 \leq \displaystyle{\frac{16a}{7}} \iff 7a^2 - 30a + 7 \leq 0 \implies a \geq \displaystyle{\frac{15 - \sqrt{176}}{7}}$.
Finally, $\displaystyle{\int_0^1{(f(x))^2dx} \geq \frac{15 - \sqrt{176}}{7} > 0,24 > \frac{3}{13}}$, so we are done.