Let $f:\mathbb{Q}^+\to \mathbb{Q}^+$ satisfies the following equation $$ f\Big(y\,\big(f(x)\big)^2\Big)=x^3\,f(xy)$$ for all positive rationals $x,y$. Show that $f(x)=\dfrac1x$ for all $x\in\mathbb{Q}^+$. Show that there are infinitely many functions $f:\mathbb{R}\to \mathbb{R}$ satisfying the above equation.
I proved that $f(1)=1$ and if $f(2)=a$ then $f(a^{2n}/8^n)=8^n$ for all integers $n$ and I get no approach here on. Can anyone help?
On
Now, I shall deal with the second part of the problem. I put this as a separate answer as MathJax is unbearably slow. In this answer, $f:\mathbb{R}\to\mathbb{R}$ is assumed.
Pick a Hamel basis $\mathcal{H}$ of $\mathbb{R}$ over $\mathbb{Q}$. Let $S$ be an arbitrary nonempty subset of $\mathcal{H}$. Write $T:=\mathcal{H}\setminus S$. Choose an arbitrary function $\sigma:T\to S$. This map $\sigma$ can be extended linearly over $\mathbb{Q}$ to obtain a $\mathbb{Q}$-linear map $\tau:\text{span}_\mathbb{Q}(T)\to\text{span}_\mathbb{Q}(S)$. Then, we define a projection $p:\mathbb{R}\to \text{span}_\mathbb{Q}(S)$, via $$p(v+w):=\tau(v)+w$$ for all $v\in\text{span}_\mathbb{Q}(T)$ and $w\in\text{span}_\mathbb{Q}(S)$. Define the function $g:\mathbb{R}\to\mathbb{R}$ by $$g(x):=\left\{ \begin{array}{ll} \exp\Big(\frac{5}{2}\,p\big(\ln(|x|)\big)\Big)&\text{if }x\neq 0\,, \\ 0&\text{if }x=0\,. \end{array}\right.$$ We note that $g:\mathbb{R}\to\mathbb{R}$ is a multiplicative function such that $$g\big(g(x)\big)=\big(g(x)\big)^{\frac{5}{2}}$$ for all $x\in\mathbb{R}$. Then, we set $$f(x):=\left\{ \begin{array}{ll} \frac{g(x)}{x}&\text{if }x\neq 0\,,\\ 0&\text{if }x=0\,. \end{array} \right.$$
We shall now show that $f$ satisfies the functional equation. Recall that $g$ is multiplicative, and so $$\begin{align} f\Big(y\,\big(f(x)\big)^2\Big)&=f\left(\frac{y}{x^2}\,\big(g(x)\big)^2\right)=\frac{g\left(\frac{y}{x^2}\,\big(g(x)\big)^2\right)}{\frac{y}{x^2}\,\big(g(x)\big)^2} \\ &=\frac{x^2\,g(y)\,\Big(g\big(g(x)\big)\Big)^2}{y\,\big(g(x)\big)^4}=\frac{x^3\,g(y)\,\Big(\big(g(x)\big)^{\frac{5}{2}}\Big)^2}{xy \,\big(g(x)\big)^4} \\ &=\frac{x^3\,g(x)\,g(y)}{xy}=x^3\,\left(\frac{g(xy)}{xy}\right)=x^3\,f(xy)\,, \end{align}$$ for all $x,y\neq 0$. If $x=0$ or $y=0$, then it is obvious that $$f\Big(y\,\big(f(x)\big)^2\Big)=0=x^3\,f(xy)\,.$$ Hence, $f$ is a solution to the required functional equation.
For a fixed Hamel basis $\mathcal{H}$, the function $f$ depends strongly on the choice of $S$ and $\sigma$. We conclude that there are infinitely many solutions to the functional equation.
Remark: If $f$ is required to be continuous, then it is not difficult to show that there are only five possible such functions. First, the zero function $f\equiv 0$ is the only non-injective solution. Then, we have the function $$f(x)=\left\{\begin{array}{ll} \frac{1}{x}&\text{if }x\neq 0\\ 0&\text{if }x=0\,, \end{array}\right.$$ and the negative of this function. Finally, we have the function $$f(x)=x\sqrt{|x|}\text{ for all }x\in\mathbb{R}\,,$$ as well as its negative. (Observe that, if $f$ is a solution, then its negative $-f$ is also a solution.)
In this answer, we assume that $f:\mathbb{Q}_{>0}\to\mathbb{Q}_{>0}$. Let $P(x,y)$ denote the condition that $$f\Big(y\,\big(f(x)\big)^2\Big)=x^3\,f(xy)\,.$$ First, we shall prove that $f$ is injective. If $x$ and $y$ are positive rational numbers such that $f(x)=f(y)$, then $$x^3\,f(x)\overset{P(x,1)}{=\!=\!=}f\Big(\big(f(x)\big)^2\Big)=f\Big(\big(f(y)\big)^2\Big)\overset{P(y,1)}{=\!=\!=}y^3\,f(y)\,.$$ As $f(x)=f(y)>0$, we get $x^3=y^3$, whence $x=y$.
Observe that $$\begin{align}f\Big(\big(f(x)\big)^2\,\big(f(y)\big)^2\Big)&\overset{P\Big(y,\big(f(x)\big)^2\Big)}{=\!=\!=\!=\!=\!=\!=}y^3\,f\Big(y\,\big(f(x)\big)^2\Big) \overset{P(x,y)}{=\!=\!=}y^3\,\Big(x^3\,f(xy)\Big) \\&=(xy)^3\,f(xy) \overset{P(xy,1)}{=\!=\!=\!=}f\Big(\big(f(xy)\big)^2\Big)\,.\end{align}$$ Because $f$ is injective, we get $$\big(f(x)\big)^2\,\big(f(y)\big)^2=\big(f(xy)\big)^2\,.$$ This shows that $f$ is multiplicative, namely, $$f(xy)=f(x)\,f(y)\text{ for all }x,y\in\mathbb{Q}_{>0}\,.$$
Thus, from $P(x,1)$, we get $$\Big(f\big(f(x)\big)\Big)^2=f\left(\big(f(x)\big)^2\right)=x^3\,f(x)\text{ for every }x\in\mathbb{Q}_{>0}\,.$$ Define $g(x):=x\,f(x)$ for each $x\in\mathbb{Q}_{>0}$. Note that $$\begin{align}\Big(f\big(g(x)\big)\Big)^2&=f\left(x^2\,\big(f(x)\big)^2\right)=\big(f(x)\big)^2\,\Big(f\big(f(x)\big)\Big)^2 \\&=\big(f(x)\big)^2\,\big(x^3\,f(x)\big)=\big(x\,f(x)\big)^3=\big(g(x)\big)^3\,.\end{align}$$ Consequently, $$\Big(g\big(g(x)\big)\Big)^2=\Big(g(x)\,f\big(g(x)\big)\Big)^2=\big(g(x)\big)^2\,\big(g(x)\big)^3=\big(g(x)\big)^5\,.\tag{*}$$
We claim that $g\equiv 1$. To prove this, we define the height $\text{ht}(r)$ of a rational number $r>0$ to be the largest integer $\beta\geq 0$ such that $2^\beta$ divides $\alpha_1,\alpha_2,\ldots,\alpha_k$, if $$r=p_1^{\alpha_1}\,p_2^{\alpha_2}\,\cdots\,p_k^{\alpha_k}\,,$$ where $p_1,p_2,\ldots,p_k$ are pairwise distinct prime natural numbers and $\alpha_1,\alpha_2,\ldots,\alpha_k\in\mathbb{Z}$. We set $\text{ht}(1)$ to be $\infty$. Clearly, $\text{ht}\big(f(x)\big)\geq \text{ht}(x)$ due to multiplicativity of $f$.
To finish the proof, we assume on the contrary that $g(x)\neq 1$ for some $x\in\mathbb{Q}_{>0}$. Let $u\in\mathbb{Q}_{>0}$ be such that $g(u)\neq 1$ and that $\text{ht}\big(g(u)\big)$ is minimized. By (*), we see that $v:=g(u)$ satisfies $$\text{ht}\big(g(v)\big)=\text{ht}\big(g(u)\big)-1\,.$$ This contradicts the choice of $u$. Therefore, $u$ cannot exist, and the claim follows. This implies that $$f(x)=\frac1x\text{ for all }x\in\mathbb{Q}_{>0}\,.$$