I understand that injective objects play a central role for doing homological algebra in an abelian category, for example having enough injectives allows one to form injective resolutions which can be used to compute derived functors.
Somehow I can't shake the feeling that I'm still missing some central observation about injectives. For example, it feels like the category $\operatorname{Inj} \mathcal A$ of injectives in an abelian category $\mathcal A$ "knows a lot about" $\mathcal A$.
Somewhat relatedly, the Gabriel's spectrum of a Grothendieck category has as underlying set the isomorphism classes of indecomposable injective objects. This strikes me as a strange choice, if one views injectives merely as a technical device to compute derived functors. Could someone explain what is going on here and what I am missing?
Let $\mathcal{A}$ be an abelian category, let $\mathcal{B}$ be the full subcategory of injective objects in $\mathcal{A}$, and let $[\mathcal{B}, \textbf{Ab}]$ be the category of additive functors $\mathcal{B} \to \textbf{Ab}$. As you say, we have the following representation theorem:
Theorem. If $\mathcal{A}$ has enough injectives, then $\mathcal{A}^\textrm{op}$ is equivalent to the full subcategory of finitely presentable objects in $[\mathcal{B}, \textbf{Ab}]$.
In other words, $\mathcal{A}$ is (equivalent to) the additive category obtained by freely adding kernels to $\mathcal{B}$. In some sense, this is the 0-dimensional version of the fact that the derived category $\mathbf{D} (\mathcal{A})$ is equivalent to the chain homotopy category $\mathbf{K} (\mathcal{B})$. (In fact, $\mathcal{A}$ embeds fully faithfully in $\mathbf{D} (\mathcal{A})$, hence also in $\mathbf{K} (\mathcal{B})$.)
The theorem is an immediate consequence of the following three propositions.
Proposition 1. If $\mathcal{A}$ has enough injectives, then the evident functor $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$ defined by $A \mapsto \textrm{Hom} (A, -)$ is fully faithful.
Given a commutative diagram in $\mathcal{A}$ of the form below, $$\require{AMScd} \begin{CD} 0 @>>> A_0 @>>> I_0 @>>> J_0 \\ &&& @VVV @VVV \\ 0 @>>> A_1 @>>> I_1 @>>> J_1 \end{CD}$$ where the rows are exact, there is a unique morphism $A_0 \to A_1$ compatible with this diagram. Conversely, given $$\require{AMScd} \begin{CD} 0 @>>> A_0 @>>> I_0 @>>> J_0 \\ & @VVV \\ 0 @>>> A_1 @>>> I_1 @>>> J_1 \end{CD}$$ where the rows are exact and $I_0$ and $J_0$ are injective, there are morphisms $I_0 \to I_1$ and $J_0 \to J_1$ compatible with this diagram. Hence, the full subcategory of $\mathcal{A}$ spanned by the objects that occur as kernels of morphisms between injective objects embeds contravariantly and fully faithfully in $[\mathcal{B}, \mathbf{Ab}]$ (by the functor in question). But $\mathcal{A}$ has enough injectives, so we are done. ◼
Proposition 2. Let $A$ be an object in $\mathcal{A}$. If $\mathcal{A}$ has enough injectives, then $\textrm{Hom} (A, -)$ is a finitely presentable object in $[\mathcal{B}, \textbf{Ab}]$.
Proof. This is clear if $A$ is injective, since representable functors are finitely presentable. But the class of finitely presentable objects is closed under cokernels, and every object in $\mathcal{A}$ is a kernel of a morphism between injective objects, and $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$ is exact, so its image is contained in the class of finitely presentable objects. ◼
Proposition 3. If $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$ is fully faithful, then every finitely presentable object in $[\mathcal{B}, \textbf{Ab}]$ is (up to isomorphism) in the image of $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$.
Proof. The class of finitely presentable objects in $[\mathcal{B}, \textbf{Ab}]$ is the smallest class of objects containing the representable functors and closed under isomorphisms, finite direct sums, and cokernels. But the image of $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$ contains the representable functors, and it is closed under finite direct sums and cokernels (because $\mathcal{A}^\textrm{op} \to [\mathcal{B}, \textbf{Ab}]$ is exact by construction and fully faithful by hypothesis), so we are done. ◼