Based on guess and check I got this formula
$$\left|\frac{1}{n!}\det\begin{pmatrix} 1&1&\dots&1&1\\ \:\:\:x_{1_1}&x_{2_1}&\dots&x_{n_1}&x_{(n+1)_1}\\ \:\:\:x_{1_2}&x_{2_2}&\dots&x_{n_2}&x_{(n+1)_2}\\ \:\:\:\vdots&\vdots&&\vdots&\vdots\\ \:\:\:x_{1_{n-1}}&x_{2_{n-1}}&\dots&x_{n_{n-1}}&x_{(n+1)_{n-1}}\\ \:\:\:x_{1_n}&x_{2_n}&\dots&x_{n_n}&x_{(n+1)_n}\end{pmatrix}\right|$$
Intuitively in $\mathbb{R}^2(n=2)$ this gives areas of any triangles
And, for another example when $n=3$
Given $4$ extreme points $$(x_{1_1},x_{1_2},x_{1_3}),(x_{2_1},x_{2_2},x_{2_3}),(x_{3_1},x_{3_2},x_{3_3}),(x_{4_1},x_{4_2},x_{4_3})$$ in $\mathbb{R}^3$, the volume of this convex hull is :
$$\left|\frac{1}{3!}\det\begin{pmatrix}1&1&1&1\\ \:\:\:x_{1_1}&x_{2_1}&x_{3_1}&x_{4_1}\\ \:\:\:x_{1_2}&x_{2_2}&x_{3_2}&x_{4_2}\\ \:\:\:x_{1_3}&x_{2_3}&x_{3_3}&x_{4_3}\end{pmatrix}\right|$$
Is this formula correct $?$
I'm trying to prove it with induction, the base case is not hard varify, but i don't know how to write the inductive step, or is this easier to do with a direct prove $?$
Any help would be appreciated.
Your formulas compute correctly the volume of simplices in ${\mathbb R}^n$, and not "the volume of any convex polytope". A simplex in ${\mathbb R}^n$ is the complex hull of $n+1$ points ${\bf x}_k\in{\mathbb R}^n$. It is a triangle in ${\mathbb R}^2$ and a (nonregular) tetrahedron in ${\mathbb R}^3$.
For a general proof you may assume ${\bf x}_4={\bf 0}$ and then think of the volume of a parallelepiped $P$ with a vertex at ${\bf 0}$ and spanned by $n$ vectors ${\bf x}_k$ $(1\leq k\leq n)$ attached at ${\bf 0}$. This $P$ can be viewed as image of the unit cube (spanned by the basis vectors ${\bf e}_k$) under the linear map with the matrix $[X]$ having the ${\bf x}_k$ in its columns. The matrix $[X]$ can be written as product of elementary matrices $${\rm diag}(\lambda_1,\lambda_2,\ldots,\lambda_n), \qquad \left[\matrix{1&\lambda\cr0&1\cr0&0&1\cr\vdots\cr0&0&&&1&0\cr0&0&&&0&1\cr}\right]\ ,\quad\ldots$$ where the $\ldots$ stand for permutation matrices. Each of these elementary matrices multiplies volumina by a factor equal to the absolute value of its determinant (as is easy to verify). Therefore the same thing has to be true for $[X]$, by the multiplicativity of determinants.
In order to explain the ${1\over n!}$ in your formulas you need an induction proof.