Let $f(x):[0, 1]\rightarrow [0, 1]$ be a continuous monotone function satsfying $f(x)>x \forall x\in (0, 1)$. Show that there is a homeomorphism $V: [0, 1]\rightarrow [0, 1]$ such that $$V(f(x))=\sqrt{V(x)}$$ holds.
THIS IS A HOMEWORK PROBLEM. I have thought about for quite some time tho.
It is not true.
Let $f$ be any function satisfying your requirements and such that $f(0) > 0$. An example is $f(x) = (x+1)/2$. This is a strictly increasing continuous function such that for $x < 1$ $$f(x) > (x+x)/2 = x .$$
Assume that there exists $V$. Then we must have either $V(0) = 0$ or $V(0) = 1$. In both case we see that $\sqrt{V(0)} = V(0)$. But then $$V(f(0)) = \sqrt{V(0)} = V(0)$$ which implies $f(0) = 0$ because $V$ is a homeomorphism. This is a contradiction.
Edited:
Paul Sinclair asked if the results holds if we additionally require $f(0) = 0$. If we use the standard interpretation of "monotonic" (see https://en.wikipedia.org/wiki/Monotonic_function), then the answer is again "no". Take any $f$ which is not strictly monotonic. An example is $$f(x) = \begin{cases} 3x & x \le 1/4 \\ 3/4 & 1/4 \le x \le 1/2 \\ (x + 1)/2 & x \ge 1/2 \end{cases}$$ This is an increasing continuous function as required. But we have $f(a) = f(b)$ for some points $0 \le a < b \le 1$ and therefore $$\sqrt{V(a)} = V(f(a) = V(f(b)) = \sqrt{V(b)}$$ which is again a contradiction.
Thus, the question remains open for strictly monotonic $f$ with $f(0) = 0$.
Edited:
If $V$ exists, we get $f(x) = V^{-1}\left(\sqrt{V(x)} \right)$. So let us see what can be said about the function $h_V = V^{-1} \circ \sqrt{\phantom {x}} \circ V$ if $V$ is a homeomorphism. Since $\sqrt{\phantom {x}} : [0,1] \to [0,1]$ is a homeomorphism, also $h_V$ is one. Moreover, since $V$ is a homeomorphism we must have either (1) $V(0) = 0, V(1) = 1$ or (2) $V(0) = 1, V(1) = 0$. In both cases we get $h_V(0) = 0, h_V(1) = 1$. Thus $h_V$ is a strictly increasing continuous function.
In case (1) $V$ is strictly increasing and so is $V^{-1}$. Thus for $x \in (0,1)$ we get $h_V(x) = V^{-1}(\sqrt{V(x)}) > V^{-1}(V(x)) = x$ because $\sqrt{y} > y$ for $y \in (0,1)$. Thus necessary conditions for $f = h_V$ are
(a) $f$ is a strictly increasing continuous function,
(b) $f(0) = 0, f(1) = 1$,
(c) $f(x) > x$ for $x \in (0,1)$.
Note that the requirement $f(1) = 1$ is redundant because (c) implies $f(1) = \lim_{x \to 1} f(x) \ge \lim_{x \to 1} x = 1$.
Similarly, in case (2) we get as necessary conditions
(a) $f$ is a strictly increasing continuous function,
(b) $f(0) = 0, f(1) = 1$,
(c) $f(x) < x$ for $x \in (0,1)$.
Now $f(0) = 0$ is redundant.
Let us finally consider the homeomorphism $V(x) = x^\alpha$ with $\alpha > 0$. We have $V^{-1}(x) = x^{1/\alpha}$ which shows that $$h_V(x) = \sqrt{x} .$$ Therefore possible solutions $V$ of $f = h_V$ cannot be expected to be unique.