I have read several posts on road maps for learning modern algebraic geometry, but it seems that whenever I open up a book on the subject (Hartshorne or Vakil's notes or Qing Liu) I make no progress whatsoever or I hit some bunp early on where I cannot proceed. I feel like this shouldn't be the case as I should have the necessary background (courses I took):
Algebra (groups, rings, fields, Sylow Theorems, Isomorphism Theorems; basically a standard course right before Galois Theory)
Commutative algebra (based on Atiyah)
Topology (Munkres)
Intro to Algebraic Geometry (classical algebraic geometry over a fixed algebraically closed field; basically chapter 1 of Hartshorne)
The main issue that I have for instance with Vakil is that I end up going down a rabbit hole. Let's say he mentions something about categories, then I go and read his section or something in a category theory book and by the time I learned that several hours passed and I forgot why I ventured away. It feels like I have to keep so many definitions and concepts in my head to just progress 2-3 lines in Vakil's notes. Another instance is that I made it to the definition of an affine scheme and I had to look up again what precisely was the meaning of a ringed space. How should I tackle this issue of keeping breadth of data in my head? Also an issue with Vakil's notes is that a lot of his motivation and intuition he uses comes from manifolds which I know nothing about or that it's difficult to screen his text for information that is important.
With Hartshorne it's even tougher. Are there perhaps other references that might be better suited for me? Or should I just progress this slowly and always keep algebra, topology, category theory books at hand and look everything up as needed?