Find all functions $f:\mathbb{R} \rightarrow \mathbb{R}$ so that:
a) $[f(x)+f(y)][f(x+2y)+f(y)]=[f(x+y)]^2+f(2y)f(y)$
b) for every real $a>b\ge 0$ we have $f(a)>f(b)$
As much as I know:
putting $x=y=0$ we get $f(0)=0$.
let $x+y=0$, the we get $f(y)+f(-y)=\frac{f(2y)}{2}$
Step 1: Prove : $f(y) = 0$ iff. $y=0$
It's easy to see $f(y)>0$ for $y>0$ (from porperty (b) with value $a=y,b=0$)
$ \ $
Step 2: Prove $f(y)$ is even.
Put $x+y=0$ in equation given to get:$f(y)+f(-y)=\frac{f(2y)}{2}$ for all y. $f(y)+f(-y)=\frac{f(2y)}{2} = f(-y) +f(y) = \frac{f(-2y)}{2} $ Thus $f(x) $is a even function, so $f(y)+f(-y)=2f(y)=\frac{f(2y)}{2} \Rightarrow f(2y)=4f(y) $ (Since $f(x)$ is even ,property (b) actually shows that $f(y) > 0$ on $(-\infty,0)\cup(0,\infty)$)
$ \ $
Step 3: Prove:$ f(y)=y^2f(1)$
By induction , find $f(ny)=n^2f(y)$ where $n \in \mathbb N$
Details of induction: Suppose $m\in \mathbb N ,m\ge 2 ,\ f(ny)=n^2f(y)$ holds for all $n\le m$
Put $x=(m-1)y$ in property (a).$\\ \Rightarrow ((m-1)^2+1)f(y)f((m+1)y)=(m^4+4-(m-1)^2-1)f^2(y) \\ \Rightarrow f((m+1)y)=(m+1)^2f(y) \ for\ y\not = 0 \ and \ it\ still\ holds\ for\ y=0$
Hint: $(m+1)^2 ((m-1)^2 +1) = (m^2-1)^2+(m+1)^2=m^4 -m^2+2 =m^4+4-(m-1)^2-1$
Remark: Result induces $f(zy)=z^2f(y)$ where $z\in \mathbb Z$ , since $f$ is even.
$ \ $
$\forall r\in \mathbb Q , \ r = \frac ab \ , \ a,b\in \mathbb Z$ . Then $f(ry)=f(\frac ab y)=a^2f(\frac 1b y)$ and $f(y)=b^2f(\frac 1b y)$
Hence $f(ry)=f(\frac ab y)=(\frac ab)^2f(y)=r^2f(y)\Rightarrow f(r)=r^2f(1)$
And $f(r)$ is even, so it's strictly decreasing for $r\le 0$.
Conclude $f(x)=x^2f(1)$ where $x\in \mathbb R $
Thus $f(x)$ can only be the form such as $ax^2$ where $a\in \mathbb R^+$