I think to let $b=[f(x_1) \dots f(x_m)]^T$ and $a=[\cos(x_1) \dots \cos(x_m)]^T$ s.t.
- a
$$h(c) := \sum_{k=1}^{m} (f(x_k)-c \cos(x_k))^2 = (b-ca)^{T}(b-ca)$$
Then expanding and setting $h'(c)=0$, we get $$c=\frac{b^Ta}{a^Ta}$$
if $a^Ta \ne 0$.
Is that right?
What if $a^Ta = 0$? I guess $a=0$ then $h(c)$ is constant. Hence, any $c$ gives the constant value $\sum_{k=1}^{m} (f(x_k))^2$. $$\therefore, c=\frac{b^Ta}{a^Ta}1_{a^Ta \ne 0} + \text{(anything)}1_{a^Ta = 0}$$
What's the role of $\cos$ here? Does it somehow prevent $a^Ta = 0$?
Possibly related:
Given two points $p_1,p_2$ both pertaining to $\mathbb{R}^m$ we have
$$ \min_{c}||p_1-c p_2||^2 = \min_{c}\left(||p_1||^2-2c\left<p_1,p_2\right>+c^2||p_2||^2\right) $$
so deriving, the stationary point is given by the solution of
$$ -2\left<p_1,p_2\right>+2c||p_2||^2=0 $$
or
$$ c = \frac{\left<p_1,p_2\right>}{||p_2||^2} $$