Need help understanding this rotation matrix math code.

Question

private Mat4 calcLookAtMatrix(Vec3 cameraPt, Vec3 lookPt, Vec3 upPt) {
    Vec3 lookDir = Glm.normalize(Vec3.sub(lookPt, cameraPt));
    Vec3 upDir = Glm.normalize(upPt);

    Vec3 rightDir = Glm.normalize(Glm.cross(lookDir, upDir));
    Vec3 perpUpDir = Glm.cross(rightDir, lookDir);

    Mat4 rotMat = new Mat4(1.0f);
    rotMat.setColumn(0, new Vec4(rightDir, 0.0f));
    rotMat.setColumn(1, new Vec4(perpUpDir, 0.0f));
    rotMat.setColumn(2, new Vec4(Vec3.negate(lookDir), 0.0f));

    rotMat = Glm.transpose(rotMat);

    Mat4 transMat = new Mat4(1.0f);
    transMat.setColumn(3, new Vec4(Vec3.negate(cameraPt), 1.0f));

    return rotMat.mul(transMat);
}

This code generates a matrix that transforms world space into camera space.

I understand everything except how the rotation matrix is formed. Basically, it puts in normalized vectors and transposes the matrix, and somehow that comes out with a rotation matrix. Explain that math.

Answer 1

The matrix is made of normalised vectors that are orthogonal, therefore its transpose is also its inverse. Assuming the camera coordinates are written in the world basis, this would yield the transformation matrix allowing you to transform world coordinates into camera coordinates.

If $V$ is the vector of coordinates in the world basis, and $C$ the vector of coordinates in the camera basis, then

$$V=BC$$

where $B$ is the matrix made of the coordinates (world basis), of the camera basis. Therefore

$$C=B^{-1}V$$

and if i'm not mistaken that's exactly what's being done at the end with rotMat.mul(transMat);, since $B^{-1}=B^T$ here (http://en.wikipedia.org/wiki/Orthogonal_matrix)