We have three linearly independent vectors $v_1=(v_{11} , ... , v_{1n})$ , $v_2=(v_{21} , ... , v_{2n})$ , $v_3=(v_{31} , ... , v_{3n})$. We want to calculate the 3D volume of the Parallelepiped made by these three vectors.
I know if we had $n$ vectors, then the $n$-d volume of these vectors is $|det[v_1,...,v_n]|$.
But we don't have $det$ of a $3 \times n$ matrix.
In general how to calculate $m$-d volume of $m$ linearly independent vectors in $n$-d space where $(m <n)$ ?
On
Claim: If $A$ is an $m \times n$ matrix with $m < n$, then the volume spanned by the rows of $A$ will be $\sqrt{\det(AA^T)}$.
Proof: Per kccu's argument (or if you prefer, using a $QR$ decomposition), there exists an orthogonal matrix $U$ such that $$ AU = \pmatrix{B & 0} $$ where $B$ is $m \times m$. For this $B$, $|\det(B)|$ is the volume that we want.
On the other hand, we have $$ |\det(B)| = \sqrt{\det(BB^T)} = \sqrt{\det[(AU)(AU)^T]} = \sqrt{\det[AUU^TA^T]} = \sqrt{\det(AA^T)} $$ as desired.
With the Cauchy-Binet formula, we could calculate this as $$ \det(AA^T) = \sum_{S \subset \{1,\dots,n\}} \det(A_S)^2 $$ where $A_S$ denotes the $m \times m$ matrix whose columns are the $i$th column of $A$ for all $i \in S$.
Find an orthonormal basis for $\mathbb{R}^n$ such that the first $3$ vectors are a basis for $\text{span}(v_1,v_2,v_3)$. Then write your vectors in terms of this orthonormal basis, and note that the last $n-3$ coordinates will all be zeros, so they can be viewed as vectors with only $3$ coordinates. Then your determinant formula works.