The question goes like this:
Find all functions $f:\mathbb{R}\rightarrow\mathbb{R}$ such that, for all $x,y\in\mathbb{R}$, $$f(xf(y)+x)=f(x)^2+xy$$
My Attempt so far
By setting $x=0$ we get $$f(0)=f(0)^2$$ Then either $f(0)=0$ or $f(0)=1$. Lets consider $f(0)=0$. Then by setting $y=0$ we get $$f(xf(0)+x)=f(x)^2$$ $$f(x)=f(x)^2$$ Then either $f(x)=0$ or $f(x)=1$. Plugging these in we see that none of them work.
Thus $f(0)=1$. Again by setting $y=0$ we get $$f(2x)=f(x)^2$$ And this is where I get stuck.
I know that $C^x$ satisfies $f(2x)=f(x)^2$, but I'm pretty sure it doesn't satisfy $f(xf(y)+x)=f(x)^2+xy$ when $y\neq0$. I know that there are other functions that satisfy $f(2x)=f(x)^2$, but I have no idea how to figure out which one will satisfy $f(xf(y)+x)=f(x)^2+xy$. I tried substituting $f(2x)=f(x)^2$, but that seemed to lead to nowhere.
I would appreciate receiving hints on how to continue.
As you say, $f(0) \in \{0,1\}$.
We have $f(f(y)+1)=f(1)^2 + y$. This implies immediately that $f$ is surjective. In particular, we may choose $u \in \mathbb{R}$ with $f(u)=-1$. Then $f(0)=f(x f(u) + x) = f(x)^2 + ux$, hence $f(0) - ux \geq 0$ for all $x \in \mathbb{R}$. This can only be the case when $u=0$ (there is no linear function which is always $\geq 0$, except when it is constant). But then $f(0)=-1$ is a contradiction.
This proves that there is no such $f$.