Suppose $f: {\bf R}^n \to {\bf R}^n $be a continuous function such that $\int_{{\bf R}^n} \vert f(x) \vert \, dx < \infty$. Let $ A \in GL_n({\bf R})$. Show that $$ \int_{{\bf R}^n} f(Ax) e^{i\langle y,x\rangle} \, dx = \int_{{\bf R}^n} f(x) e^{i\langle (A^{-1})^ty,x\rangle} \frac {dx}{\vert \det(A)\vert }.$$
Here,$<x,y>$ denote standard inner product on $\bf R^n$.Clearly using $Ax=t$ in the L.H.S. gives the R.H.S expression except $\vert \det(A)\vert ^{-1}$. I guess this $\vert \det(A)\vert^ {-1} $ is Jacobian of Matrix of change of variable but unable to verify this. Please help.
The change of variable you need to make is $x = A^{-1}z$.
By definition of matrix-vector multiplication, we have $x_i = \displaystyle\sum_{j = 1}^{n}(A^{-1})_{i,j}z_j$.
Hence, the $(i,j)$-th entry of the Jacobian matrix is $\dfrac{\partial x_i}{\partial z_j} = (A^{-1})_{i,j}$.
Therefore, the Jacobian matrix is simply $A^{-1}$, and its determinant is $\det (A^{-1}) = (\det A)^{-1}$.
Thus, the change of variables gives $\displaystyle\int_{\mathbb{R}^n}f(Ax)e^{i\langle y, x \rangle}\,dx = \displaystyle\int_{\mathbb{R}^n}f(z)e^{i\langle y, A^{-1}z \rangle}|\det A|^{-1}\,dz$.
Then, by using $\langle y, A^{-1}z \rangle = \langle (A^{-1})^Ty, z \rangle$, you get the desired result.