I had a doubt regarding Lemma 7.3 and Lemma 7.4 from the third chapter of Hartshorne in the section on the Serre Duality Theorem.
In Lemma 7.3 he proves that if $X$ is a closed subscheme of codimension $r$ of $P=\mathbb{P}^N_k$ (for a field $k$) then $\mathcal{Ext}^i_P(\mathcal{O}_X,\omega_P)=0$. I do not understand the statement because $\mathcal{O}_X$ is not a defined over the space of $P$, so talking about $\mathcal{Hom}_P(\mathcal{O}_X,\cdot)$ does not make any sense to me. Going back to Remark 2.10.1 in the same chapter, he says that they sometimes use the notation $\mathcal{F}$ instead of $j_*\mathcal{F}$ (where $j$ is an inclusion map of a closed set to the original space), I am guessing he means the same thing here, but I am not sure (It would be great if someone could clarify). I then moved ahead trying to understand the proof by just assuming what he denotes as $\mathcal{F}_i$ to be some coherent sheaf. He says that to show that $\mathcal{F}_i=0$, all we need to do is show that for all large enough $q$, $\Gamma(P,\mathcal{F}(q))=0$. He uses Theorem 5.17 from Chapter 2 for this. I am unsure how that works. I personally feel that we would have to use a combination of Theorem 5.15 and Excercise 5.9(c) (again from Chapter 2) to prove this. Am I wrong? Is Hartshorne trying to suggest a different proof?
Now moving onto Lemma 7.4. Here I do not understand how $\mathcal{Ext}^r_P(\mathcal{O}_X, \omega_P)$ is an $\mathcal{O}_X$-module and how $\mathcal{F}$ which he takes as an $\mathcal{O}_X$-module is then considered as a $\mathcal{O}_P$-module! Is he using the operation $j^*$ and $j_*$ respectively, but not explicitly mentioning it?
Thanks in advance to anyone who doesn't mind taking the time out to answer this.
For your first question, you are correct that they mean $j_*\mathcal{O}_X$ for $j:X\to P$ the closed immersion.
For the second question, recall that Theorem II.5.17 says that for a coherent sheaf $\mathcal{F}$ on a projective scheme over a noetherian ring, $\mathcal{F}(q)$ is generated by global sections for all $q>>0$. If a sheaf $\mathcal{F}$ is generated by global sections and also has no global sections, then it must be zero, so Hartshorne's claim works. Your alternative proof is also valid.
Finally, the answer to your third question is yes. Recall that one can compute $\mathcal{Ext}_P(j_*\mathcal{O}_X,\omega_P)$ by taking a resolution of $\omega_P$, then applying the functor $\mathcal{Hom}_P(j_*\mathcal{O}_X,-)$ to this sequence and finally taking cohomology. But any sheaf on $P$ which is can be written as $\mathcal{Hom}_p(j_*\mathcal{O}_X,-)$ is supported on $X\subset \Bbb P^n$, so applying $j^*$ will turn this in to a sheaf on $X$ and do what you want. The fancy words here are that the functors $j_*$ and $j^*$ define an exact fully-faithful equivalence of categories between $\mathcal{O}_X$ modules and $\mathcal{O}_P$ modules supported on $X$.