Consider problem III.5.2(a) in Hartshorne's Algebraic Geometry:
Let $X$ be a projective scheme over a field $k$, let $\mathcal O_X(1)$ be a very ample invertible sheaf on $X$ over $k$, and let $\mathcal F$ be a coherent sheaf on $X$. Show that there is a polynomial $P(z)\in \mathbb Q[z]$, such that $\chi(\mathcal F(n))=P(n)$ for all $n\in\mathbb Z$. We call $P$ the Hilbert polynomial of $\mathcal F$ with respect to the sheaf $\mathcal O_X(1)$. [Hints: Use induction on dim Supp $\mathcal F$, general properties of numerical polynomials (I, 7.3), and suitable exact sequences
$$0\rightarrow \mathcal R \rightarrow \mathcal F(-1)\rightarrow \mathcal F \rightarrow \mathcal L \rightarrow 0.]$$
By III.2.10, we may suppose $X=\mathbb P^n_k$ for some $n$.
I think the key difficulty here is describing an injective map $\alpha:\mathcal F(-1)\rightarrow \mathcal F$. For any map between these two sheaves, we get an exact sequence of the kind in the hint by taking the relevant quotient and kernel sheaves. If this map is injective, we get a short exact sequence, and we can use the additivity of the Euler characteristic on short exact sequences to say $$\chi(\mathcal F) - \chi(\mathcal F(-1))= \chi (\mathcal L).$$ Tensoring with $\mathcal O(n)$ gives $$\chi(\mathcal F(n)) - \chi(\mathcal F(n-1))= \chi (\mathcal L(n)).$$ So if $\chi(L(n))$ is eventually a polynomial, then $\mathcal F(n)$ is too, by I.7.3(b). If we construct $\alpha$ in such a way that $\mathcal L$ has support at least one dimension less than the support of $\mathcal F$, we're done by induction on the dimension of the support. (We also need to handle dimension $0$ separately, and I omit this discussion because it's not too hard.)
So, let me say what I think $\alpha$ should be.
Consider a global section $s$ of $O(1)$. Tensoring with $s$ gives a map $\mathcal F(-1)\rightarrow \mathcal F$. In some affine chart, this looks like multiplication by $s$, I think. Now we examine the stalks of the quotient sheaf. The first map, on the level of stalks, is an isomorphism when $s\mathcal F_p=\mathcal F_p$, which happens in particular when $s$ is a unit in the local ring $\mathcal O_{X,p}$, and this happens when $s_p \notin \mathfrak m_p$. Then the support of $\mathcal L$ is contained in the intersection of the support of $\mathcal F$ and $V(s)$, where we consider $s$ as a linear polynomial over $k$. So clearly the dimension of the support goes down by at least one, since hyperplanes have codimension $1$.
It seems tricker to choose $s$ so that $\alpha$ is injective. By II.5.15, we know that every (quasi)coherent sheaf on projective space over a field is given as $\tilde M$ for some module $M$ over $k[x_i]$. We need $s$ to not be a zero-divisor. Recall that the set of zero-divisors is exactly the union of the associated primes, and that there are finitely many associated primes. But primes represent subvarieties, and these subvarieties might be impossible to avoid. For example, if a few hyperplanes are among the associate primes, then there’s no way we can choose $\alpha$ to be injective. Is this analysis correct?
If so, it seems we need to analyze $\mathcal R$ more carefully and not just suppose it disappears. Could we say something like, its support lies on the union of the associated primes, which are all subvarieties of codimension at least 1, so we are finished by an argument similar to the one indicated above? Explicitly, we would have
$$\chi(\mathcal F(n)) - \chi(\mathcal F(n-1))= \chi (\mathcal L(n)) - \chi (\mathcal R(n)),$$ and then we can induct on dimension and apply I.7.3(b).
I don't think you need an injective map $\mathcal{F}(-1)\to \mathcal{F}$. The point is that since $\chi$ is additive, then for any exact sequence $$0\to \mathcal{F}_1\to\mathcal{F}_2\to...\to \mathcal{F}_n\to 0$$ we have $\sum_{i=1}^n (-1)^i\chi(\mathcal{F}_i)=0$. Just split the long exact sequence into many short exact sequences. With this in mind here is my attempt at a solution:
We know $Z:=\operatorname{Supp}(\mathcal{F})\subset \mathbb{P}^n$ is a closed subset. Let $L\subset \mathbb{P}^n$ be a hyperplane which does not contain any irreducible component of $\mathcal{F}$. Then $L$ corresponds to some section $s\in \mathcal{O}(1)$ and it gives rise to a map $\mathcal{F}(-1)\to \mathcal{F}$. Now consider an exact sequence $$0\to \mathcal{R}\to \mathcal{F}(-1)\to \mathcal{F}\to \mathcal{L}\to 0.$$ We have $\operatorname{Supp}(\mathcal{L}),\operatorname{Supp}(\mathcal{R})\subset \operatorname{Supp}(\mathcal{F})\cap V(s)$ because on $D_+(s)$ the map $\cdot s$ is an isomorphism. Therefore both have dimension strictly less than $\dim\operatorname{Supp}(\mathcal{F})$. By induction we can find polynomials $P_\mathcal{R}(z), P_\mathcal{L}(z)$ such that $$P_\mathcal{R}(n)=\chi(\mathcal{R}(n)), \ P_\mathcal{L}(n)=\chi(\mathcal{L}(n))$$ for all $n\in \mathbb{Z}$. We then get, for any $n\in \mathbb{Z}$, $$\chi(\mathcal{F}(n))-\chi(\mathcal{F}(n-1))=P_\mathcal{L}(n)-P_\mathcal{R}(n).$$ It then follows from the proof of proposition I.7.3 in Hartshorne that there is a numerical polynomial $P_\mathcal{F}(z)$ such that $P_\mathcal{F}(n)=\chi(\mathcal{F}(n))$ for all $n$.