The matrix of a rotation of a point, $(x,y)$ in cartesian coordinates, counterclockwise is $$\begin{pmatrix}cosθ & -sinθ \\ sinθ & cosθ\end{pmatrix}$$
When rotating a coordinate system counterclockwise, instead of a single point, do we use the same matrix?
(The reason I'm asking this is because I've seen in some books that the matrix for coordinate system transformation be the transpose of this. And this has caused a lot of confusion.)
When you rotate a coordinate system, what you're really doing is rotating the basis vectors of the coordinate system. These are just individual vectors like any other vector you want to rotate, so the transformation matrices are unchanged.
Do be careful of the direction of rotation though! For counterclockwise it's: \begin{align*} R = \begin{pmatrix}\cos\theta &-\sin\theta\\ \sin\theta &\cos\theta\end{pmatrix} \end{align*} The clockwise rotation through $\theta$ must be the inverse of this matrix, because composing the two matrices will give zero rotation (and hence the identity matrix). Thus for clockwise, \begin{align*} R = \begin{pmatrix}\cos\theta &-\sin\theta\\ \sin\theta &\cos\theta\end{pmatrix}^{-1} = \begin{pmatrix}\cos\theta &\sin\theta\\ -\sin\theta &\cos\theta\end{pmatrix} \end{align*}