Problem: Fix $a, b \in \mathbb{R}$ with $a > b > 0$. Find $f, g, h: \mathbb{R} \to \mathbb{R}$ such that $$ 2f(x) g(y) + f(x)^2 + g(y)^2 = 2 \sqrt{a} \ h(x+y) + (a+b) \\ 2 f(x) g(y) - f(x)^2 - g(y)^2 = 2 \sqrt{a} \ h(x-y) - (a + b) $$ It's also possible that $f = g = h$.
Attempt. I tried to consider $f(x) = \cos(x)$ and $g(y) = \sqrt{a}\cos(y)$ which would satisfy the addition of both these equations given by $$ 2 f(x) g(y) = \sqrt{a} \left( h(x+y) + h(x-y) \right) $$ with $h(z) = \cos(z)$ as well due to product-to-sum identities (i.e. $\cos$ and $\cosh$ satisfy d'Alembert's equation) but they won't necessarily satisfy each equation before addition..
Any ideas?
We have $$ (f(x) +g(y))^2 = 2 \sqrt{a} \ h(x+y) + (a+b) \\ -( f(x) -g(y))^2 = 2 \sqrt{a} \ h(x-y) - (a + b)$$
$$ f(x) +g(y) = \sqrt{2 \sqrt{a} \ h(x+y) + (a+b)} \\ f(x) -g(y) = \sqrt{-2 \sqrt{a} \ h(x-y) + (a + b)} $$ Add and subtract: $$ 2f(x) = \sqrt{2 \sqrt{a} \ h(x+y) + (a+b)} + \sqrt{-2 \sqrt{a} \ h(x-y) + (a + b)} \tag5$$ $$2g(y) = \sqrt{2 \sqrt{a} \ h(x+y) + (a+b)} - \sqrt{-2 \sqrt{a} \ h(x-y) + (a + b)}\tag 6 $$ Put y=0 in (6): $$2g(0) = \sqrt{2 \sqrt{a} \ h(x) + (a+b)} - \sqrt{-2 \sqrt{a} \ h(x) + (a + b)} \tag7$$ Assuming $a\ne0$, this means that h(x) must be constant for all x. So we have $$ 2f(x) = \sqrt{2 \sqrt{a} \ h + (a+b)} + \sqrt{-2 \sqrt{a} \ h + (a + b)} \tag8$$ $$2g(y) = \sqrt{2 \sqrt{a} \ h + (a+b)} - \sqrt{-2 \sqrt{a} \ h + (a + b)}\tag 9 $$ and so f,g,h are all constant functions.