Find all functions such that $f(x)f(y) = f(xy + 1) + f(x - y) – 2$ for all $x, y $ are real numbers.
I put y=0 into the equation and get $(f(0)-1)f(x)=f(1)-2$. If $f(0)≠1$, then $f(x) = (f(1)-2)/(f(0)-1)$ and so the function would be constant. But, since $c^2=2c-2$ has no real solutions, the function cannot be constant. Hence, $f(0)=1$, which implies that $f(1)=2$. Also note that f must be an even function as $f(x-y)=f(y-x)$.
Set $f(x)=1+x^2+g(x)$ for some even function of $g$, with $g(0)=g(1)=0$. Then put $f(x)$ into the functional equation and substitute $y=1$, get $g(x+1)-g(x)=g(x)-g(x-1)$. Define $P(x) = {…, g(x-2), g(x-1), g(x), g(x+1), g(x+2),…}$ , so $P(x)$ is an A.P. Then substitute $y = -x , y=x$ and using the fact that g is even, to get $g(x^2+1) – g(x^2-1) = g(2x)$. I’m trying to prove $g(x)=0$ for all $x$. How can I proceed?
Let $S=\{\,x\in \Bbb R\mid f(x)=x^2+1\,\}$. You already know: $f(0)=1$, $f(1)=2$, $f$ is even, i.e., $\{0,1\}\subset S$ and $S=-S$. We want to show $S=\Bbb R$, and this is equivalent to your endeavour to show $g\equiv 0$, but I find it easier to think of it this way.
One readily sees from the fact that $x\mapsto x^2+1$ solves the functional equation:
$$\tag0\text{If three out of }x,y,xy+1,x-y\text{ are }\in S\text{, then so is the fourth.}$$
With $y=1\in S$, this becomes $$ \tag1\text {If two of }x-1,x,x+1\text{ are }\in S\text{, then so is the third.}$$ From this and $0,1\in S$, induction gives us $$\Bbb Z\subset S.$$ Likewise, with $x=y$ and using $0\in S$, $(0)$ becomes $$ \tag2\text{If }x\in S\text{, then }x^2+1\in S.$$ More precisely, $y=x$ gives us $$ \tag 3f(x)^2+1=f(x^2+1),$$ and so $$\tag4 f(x)\ge 1\quad\text{for }|x|\ge 1.$$ Consider $x$ with $0<|x|<1$. With $y=2\operatorname{sgn}(x)+x$, we get that $|y|>1$, $|xy+1|>1$, $y-x=\pm2$ so that by $(4)$, $f(x)f(y)=f(xy+1)+3\ge 4$, and hence $f(x)>0$. Thus together with $(4)$ and $f(0)=1$, $$ f(x)>0\qquad\text{for all }x\in\Bbb R.$$ This and $(3)$ allows us to improve $(2)$ to $$ \tag5x\in S\iff x^2+1\in S.$$ For $0\le u_0<v_0$, let $u_n=u_{n-1}^2+1$, $v_n=v_{n-1}^2+1$. Then $u_n<v_n$, $(u_{n-1},v_{n-1})$ is mapped bijectively to $(u_n,v_n)$ by $x\mapsto x^2+1$, and $$v_n-u_n=(u_{n-1}+v_{n-1})(v_{n-1}-u_{n-1})>2u_{n-1}(v_{n-1}-u_{n-1})\ge 2(v_{n-1}-u_{n-1})$$ for $n\ge 2$, hence some interval $(u_n,v_n)$ will have length $>1$ and therefore intersect $\Bbb Z$ and also $S$. We conclude from $(5)$ that also $(u_0,v_0)$ intersects $S$, hence $$ \overline S=\Bbb R.$$ In other words,