Let $Y$ be a noetherian scheme, and let $\mathcal E$ be a locally free $\mathcal O_Y$-module of rank $n+1$, with $n\ge 1$. Let $X=\mathbb P(\mathcal E)$ [the projective bundle over $\mathcal E$], with corresponding invertible sheaf $O_X(1)$ and projection morphism $\pi:X\rightarrow Y$.
Exercise III.8.4(c) in Hartshorne says
Now show, for any $l\in\mathbb Z$, that $$R^n\pi_*(O(l))\cong \pi_*(O(-l-n-1))^\vee \otimes (\wedge^{n+1} \mathcal E)^\vee.$$
I interpret "Now show" to mean 'Use the previous parts of the exercise to help you show this', so it is worth noting that in part (a), we showed that $R^i\pi_*(O(l))$ vanishes for most values of $i$ and $l$, and in (b), $$R^n\pi_*((\pi^*\wedge^{n+1} \mathcal E)(-n-1))\cong \mathcal O_Y.$$ Also, $\omega_{X/Y}\cong (\pi^*\wedge^{n+1} \mathcal E)(-n-1)$.
We also have the projection formula $$R^if_*(\mathcal F \otimes f^* \mathcal E)\cong R^if_*(\mathcal F)\otimes \mathcal E.$$
Using this formula, we have
$$R^n\pi_*(\mathcal O(l))\otimes (\wedge^{n+1} \mathcal E) \cong R^n \pi_*(\pi^*(\wedge^{n+1} \mathcal E) \ \otimes \mathcal O (l))$$ Twisting by $0$ inside the parentheses by adding and subtracting $n+1$ shows this is equal to $$ R^n\pi_*(\omega_{X/Y} \otimes O(n+l+1)).$$
Now, using Proposition III.8.1, it seems we might be done if had some version of Serre duality for $X/Y$ and used this on an appropriate affine cover of the base and patched things together.
However, Hartshorne is very careful to not use things he hasn't proved previously, and he has only proved Serre duality for projective schemes over a field. We have also not used the fact that $$R^n\pi_*((\pi^*\wedge^{n+1} \mathcal E)(-n-1))\cong \mathcal O_Y,$$ which seems important and useful. I conclude that I am probably approaching this the wrong way, and that some different method is needed to solve the problem with only the tools Hartshorne has developed so far.
On the other hand, I don't know how I would obtain an expression of the form $$\pi_*(O(-l-n-1))^\vee$$ without a duality theorem.
How should one proceed here?
Hartshorne does prove Serre duality for $\mathbb P^n$ over any Noetherian ring (Theorem III.5.1). Locally, $\mathbb P(\mathscr E)$ is of this form (e.g. on an affine cover that trivialises $\mathscr E$).
To check that the duality commutes with restriction (on $\operatorname{Spec} A$), observe that the pairing $$H^0(\mathbb P^n, \mathcal O(d)) \times H^n(\mathbb P^n, \mathcal O(-d-n-1) \to H^n(\mathbb P^n, \mathcal O(-n-1)) = A$$ is defined in terms of a Čech complex whose formation 'does not depend on $A$'.