I am trying to solve Hartshorne's exercise V $1.2$.
Let $H$ be a very ample divisor on the surface $X$, corresponding to a projective embedding $X\subseteq P^N$. Then the Hilbert polynomial of $X$ can be written as $P(z)=\frac{1}{2}az^2+bz+c$, where $a=H^2,b=H^2/2+\pi,c=1+p_a$. Here the Hilbert polynomial is defined as $P(n)=\mathcal{X}(\mathcal{F_H}(n))$, where $\mathcal{F}$ is the invertible sheaf corresponds to divisor $H$, $\mathcal{X}$ means the Euler characteristic, and $\mathcal{F_H}(n)=\mathcal{F_H}\otimes \mathcal{O}_X(n)$. And $H^2$ denotes the self-intersection number of $H$, $\pi$ is the genus of a nonsingular curve representing $H$, and $p_a$ means the arithmetic genus.
First, we want to find $c=P(0)$, which should be $1+p_a=\mathcal{X}(\mathcal{O}_X)$. But $P(0)=\mathcal{X}(\mathcal{F_H}(0))=\mathcal{X}(\mathcal{F_H})$. Why does it equal to $\mathcal{X}(\mathcal{O}_X)$? Besides, if we can compute $P(1)=\mathcal{X}(\mathcal{F_H}(1))$ and $P(-1)=\mathcal{X}(\mathcal{F_H}(-1))$, then we can determine the polynomial. But none of these are simple to me, so I hope someone could help.
Also, Hartshorne states that if $C$ is any curve on $X$, then its degree in $P^N$ is just $C.H$. These should mean the Hilbert polynomial of $C$ is $P(z)=(C.H)z+c$, and $c=\mathcal{X}(\mathcal{F_C}(0))$. But I still have no idea on it.
By the way, I hope someone can tell me a systematic way to compute Hilbert polynomial. Thanks!
first your definition of Hilbert polynomial isn't correct:$P(n)=\mathcal{X}(O_X(n))$ where $O_X(n)=O_X\otimes \mathcal{F}_H^n$
now $P(0)=\mathcal{X}(O_X)=1+p_a$. In general from Riemann-Roch theorem we have $$P(n)=\mathcal{X}(\mathcal{F}_H^n)=\frac{1}{2}(nH.nH-nH.K)+1+p_a$$ so you must prove that $H.K=H^2-(2\pi-2)$ which is the Adjunction formula.
for the second part note that $H\cap C$ is a very ample divisor on C with degree $C.H$ then use the Riemann-Roch theorem on the curve C.