Let $\mathbb R^+$ be the set of positive numbers. Determine all functions $f:\mathbb R^+\to\mathbb R^+$ such that $$f(x^2+xf(y)) = f(f(x))(x+y)$$ for all positive real numbers $x$ and $y$.
I think that the only solution is $f(x) = x$. But I'm not sure how to prove it. I would really appreciate any suggestions.
You are correct that the only solution is $f(x)=x$.
First we claim that $f$ must be injective. Indeed, if $f(x)=f(y)$, then we have $$f(f(x))(x+y)= f(x^2+xf(y))=f(x^2+xf(x))=f(f(x))(x+x)\text{,}$$ and canceling (since we are in $\mathbb R^+$) gives $x=y$.
But now if $x+y=1$, we have $$f(x^2+xf(y))=f(f(x))(x+y)=f(f(x))\text{,}$$ so by injectivity we get $$x^2+xf(y)=f(x)\text{,}$$ and similarly $$y^2+yf(x)=f(y)\text{.}$$ Using $y=1-x$ we can solve this system \begin{align*} x^2+x(y^2+yf(x)) &=f(x)\\ x^2+x\left((1-x)^2+(1-x)f(x)\right) &= f(x)\\ x^3-x^2+x &= f(x)(x^2-x+1)\\ x=f(x)\text{,} \end{align*} so that every $x<1$ is a fixed point of $f$.
But if $x$ is a fixed point, we also have $$f(2x^2)=f(x^2+xf(x))=f(f(x))(x+x)=2x^2\text{,}$$ and so the fixed point set includes $(0,1)$ and is closed under the map $\phi(x)=2x^2$. Since $\phi$ is continuous, and takes $0$ to $0$, the fixed point set includes $(0,\phi^n(1))$ for all $n$. Since $\phi^n(1)\to\infty$, this completes the proof.