let's consider the coordinate transformation in three-dimensional Euclidean space:
$\mathbf r=\mathbf a+\mathbf r',\qquad\mathbf r=\mathbf ix+\mathbf jy+\mathbf kz,\qquad\mathbf r'=\mathbf i'x'+\mathbf j'y'+\mathbf k'z',\qquad\mathbf a$ is a constant shift vector.
$x=a_x+x'\cos(\widehat{\mathbf i,\mathbf i'})+y'\cos(\widehat{\mathbf i,\mathbf j'})+z'\cos(\widehat{\mathbf i,\mathbf k'}),$
$y=a_y+x'\cos(\widehat{\mathbf j,\mathbf i'})+y'\cos(\widehat{\mathbf j,\mathbf j'})+z'\cos(\widehat{\mathbf j,\mathbf k'}),$
$z=a_z+z'\cos(\widehat{\mathbf k,\mathbf i'})+y'\cos(\widehat{\mathbf k,\mathbf j'})+z'\cos(\widehat{\mathbf k,\mathbf k'}).$
It seems to me that only 5 of these 9 angles are independent, namely 5 of these angles completely set the orientation of the $O'x'y'z'$ coordinate system relative to the $Oxyz$ one. However, I'm not completely sure of it, and I don't know how to prove it and find relations between all these 9 angles.
I would appreciate your advice and/or links (e.g. books).
Partial answer that will hopefully help.
The constant shift $\mathbf{a}$ is totally irrelevant as it can be absorbed into $\left(\begin{smallmatrix}x\\y\\z\end{smallmatrix}\right).$ Assuming that $(\mathbf{i,j,k})$ and $(\mathbf{i',j',k'})$ are both Cartesian a positive orientation for both is expressed by the right-hand-rule and by the cross product as $$\tag{1} \mathbf{i\times j=k}\,,\quad \mathbf{i'\times j'=k'}\,. $$ From your system of equations it follows that \begin{align}\tag{2} \mathbf{i}=\begin{pmatrix}c_{11}\\ c_{21}\\c_{31} \end{pmatrix}\,,\quad \mathbf{j}=\begin{pmatrix}c_{12}\\ c_{22}\\c_{32} \end{pmatrix}\,, \mathbf{k}=\begin{pmatrix}c_{13}\\ c_{22}\\c_{33} \end{pmatrix}\,, \end{align} where I wrote $c_{11}=\cos(\widehat{\mathbf{i,i'}})\,,c_{12}=\cos(\widehat{\mathbf{i,j'}})$ and so on. If you write out (1) using (2) you will get three more equations that relate the $c_{ij}$ to each other.