Often, the curl of a vector field $\nabla \times\vec{F}$ is described as the tendency for a twig to rotate if you place it in a flow of water described by the vector field $\vec{F}(\vec{x})$. I find this intuitive in most cases, however, there is one case where it doesn't make intuitive sense.
Say we have a vector field given by $\vec{F}(\vec{x}) = \frac{\vec{x}}{||\vec{x}||^3}$ where both $\vec{F}$ and $\vec{x}$ are 3-dimensional vectors. The curl of this vector field is zero: $\nabla \times \vec{F} = 0$
Lets say this field describes water flow and we place a twig (depicted by the green rectangle) in the water as shown in the figure (drawn in 2d). Intuitively, it feels like the twig should rotate (like the yellow rectangle in the figure) as the part of the twig closest to the origin experiences a larger force than the part furthest away, and therefore there is a torque on the twig.

The math tells me that the curl is zero, but my physics intuition tells me that there is "some rotation" in the vector field.
So, is my physics intuition wrong and the twig won't rotate, or is there something wrong with the "twig in water" analogy?
There's a little bit of both kinds of "wrong" here. The analogy is imperfect (as many analogies are) and your interpretation of it is imperfect.
I believe the twig in the analogy is supposed to be very small compared to the variation in the vector field. Your diagram is more like dropping a huge log in the water to see what happens.
Mathematically, we're looking at the limit of a phenomenon as we take the length of the twig to zero, so that the vectors it interacts with are almost the same in magnitude and direction. We shouldn't be concerned about the differences in the vectors across large distances in the field.
On the other hand, what we can learn from your log is that if you drop it in the water to the right of the $y$ axis and parallel to the $y$ axis (instead of above the $x$ axis and parallel to the $x$ axis), it will rotate counterclockwise to the orientation shown in yellow instead of clockwise. Once its midpoint is on a vector perpendicular to the log, the forces will be balanced, and if it turns farther there will be a restoring force.
The "rotating twig" analogy means that no matter how you drop the twig in the water (other than parallel to the direction of the field), it will experience a turning force in the same direction. It won't be turned clockwise if it starts in one orientation and counterclockwise if it starts in another.
On the other hand again, a flaw in the "rotating twig" analogy is that in certain orientations, the twig can still experience a turning force in one direction even though the curl of the field is zero. You have to try starting the twig in different orientations or follow the motion of the twig for a while to see that it does not consistently turn in the same direction.
I think the analogy that I have seen is if you have a tiny little paddlewheel in the vector field, will it turn? To translate this into the "twig" model, you could drop a bundle of twigs tied together like the spokes of a wagon wheel so that if one of the twigs experiences a clockwise torque, it is balanced by another twig experiencing a counterclockwise torque.