I want to prove the following.
We have a function $f: \mathbb{Z} \to\mathbb{R}$ s.t.
(1) $f(mn) = f(m)f(n)$
(2) $f(m+n) \leq f(m) + f(n)$
(3) $0 \leq f(x) \leq 1$
then $f(m+n) \leq \max\big(f(m), f(n)\big)$?
I tried to use $f(m+n)^k$ and use (1)-(3), but I could not show the inequality.
Please help me.
On
Firstly, note that $f(0)=f(0)^2$ and $f(1)=f(1)^2$, i.e. $f(0),f(1)=0$ or $1$. If $f(0)=1$, then $1=f(0)=f(n)f(0)=f(n)$ for every $n\in\mathbb{Z}$; if $f(1)=0$, then $0=f(n)f(1)=f(n)$ for every $n\in\mathbb{Z}$. For both cases, the conclusion is trivially true. Therefore, we only need to focus on the case $f(0)=0$ and $f(1)=1$.
Note that by (1) and (3), we have (4): if $m\mid n$, then $f(n)\le f(m)$. Now given any pair of prime numbers $p\ne q$, according to Fermat's little theorem, $q\mid (p^{r(q-1)}-1)$ for every $r\ge 1$. Therefore, by (1), (2) and (4):
$$1=f(1)\le f(p^{r(q-1)})+f(1-p^{r(q-1)})\le f(p)^r+f(q).$$ If $0\le f(p)<1$, letting $r\to\infty$, we can conclude that $f(q)=1$. Since $p\ne q$ are arbitrary, it implies that either for each prime number $p$, $f(p)=1$; or there exists a unique prime number $p$, $0\le f(p)<1$ and for every prime number $q\ne p$, $f(q)=1$.
Therefore, either for every $n\ne 0$, $f(n)=1$; or there exists a prime number $p$, such that $0\le f(p)<1$, and if $n=p^rm$ with $p \nmid m$, then $f(n)=f(p)^r$. The conclusion follows from this immediately.
This is not a finished solution, but maybe a stepping stone?
From 1) we get $f(m\cdot 1)=f(m)f(1)$, so either $f(1)=1$ or $f(n)=0$ for all $n$.
Assume $f(1)=1$, then 2) gives $f(n)\leq f(1+...+1)=f(1)+...+f(1)\leq n$.
In particular, $f(0)\leq f(0)+f(0)$ so $f(0)\leq 0$. Condition 3) then forces $f(0)=0$.
We also have $1=f(1)=f(-1)f(-1)$ giving $f(-1)=1$ and $f(-n)=f(n)$ for all $n$.
Now, $f(m)^2+f(n)^2\geq f(m^2-n^2)=f(m+n)f(m-n)$. Assuming $f(m-n)\neq 0$,
$f(m+n)\leq \frac{f(m)^2+f(n)^2}{f(m-n)}$.
Maybe this can be massaged into what you are after?