If $\mathfrak{so}(3)$ is the Lie algebra of $SO(3)$ then why are the matrices of $\mathfrak{so}(3)$ not rotation matrices? They aren't infinitesimal rotations either. The matrices of $\mathfrak{so}(3)$ are skew-symmetric matrices which are the type used to calculate the cross product.
How can $\mathfrak{so}(3)$ be tangent to $SO(3)$ if they're never even in $SO(3)$?
On
$A$ is an infinitesimal rotation means that $e^A$ is a rotation.
Consider $e^{tA}$, you have ${d\over{dt}}\langle e^{tA}x,e^{tA}x\rangle=\langle$ $A(e^{tA}x),e^{tA}x\rangle+\langle e^{tA}x,A(e^{tA}x)\rangle=0$ since $A$ is antisymmetric. You deduce that $\langle e^{tA}x,e^{tA}x\rangle$ is constant ant its value is $\langle e^{0A}x,e^{0A}x\rangle=\langle x,x\rangle$.
The same way $(0,\frac12)$ can be a tangent vector to the unit circle at $(1,0)$ even though $(0,\frac12)$ is not on the unit circle.
One (loose and informal!) way to think about it is that an element of $\mathfrak{so}(3)$ is the difference between the matrix of an infinitesimal rotation and the identity matrix, but "scaled up by a factor of infinity" such that the entries of the matrix don't need to be infintesimals themselves.