Let's consider the group
$$\operatorname{GL}_n(\mathbb{R})_{+} = \left\{ M \in M_n(\mathbb{R}) \mid \det(M) > 0\right\}$$
We identify this set as a open subset of $\mathbb{R}^{n^2}$. It is known that:
$$ \mu(A) := \int_A \frac{1}{\det(x)^n} \, dx $$
where $A$ is a Borel subset of $\operatorname{GL}_n(\mathbb{R})_{+}$ (and by the identification of $\mathbb{R}^{n^2}$) and $dx$ is the Lebesgue measure on $\mathbb{R}^{n^2}$ is a (left and right) Haar measure.
$\textbf{My attempt:}$ Let's see that $\mu$ is left invariant. Let $C \in GL_n(\mathbb{R})_{+}$ and $A$ a Borel subset of $GL_n(\mathbb{R})_{+}$. We have:
$$ \mu(CA) = \int_{CA} \frac{1}{\det(x)^n} \, dx $$
We have $M_n(\mathbb{R}) = \mathbb{R}^n \oplus \mathbb{R}^n \oplus \ldots \oplus \mathbb{R}^n$ viewing as column vectors the elements of $\mathbb{R}^n$. So $C=C_1 \oplus C_2 \oplus \ldots \oplus C_n$ and $M=M_1 \oplus M_2 \oplus \ldots \oplus M_n$ where $C_i$ and $M_i$ are the columns of $C$ and $M \in \operatorname{GL}_n(\mathbb{R})_+$.
Consider the linear transformation $T: M_n(\mathbb{R}) \rightarrow M_n(\mathbb{R})$ defined by $T(M)=CM$. We can see this simply as
$$ T(M_1 \oplus M_2 \oplus \ldots \oplus M_n) = CM_1 \oplus CM_2 \oplus \ldots \oplus CM_n$$
Considering the canonical basis in $\mathbb{R}^n$ it's easy to see that $\det(T)=\det(C)^n$ (see that this is the Jacobian of $T$). So we have:
$$ \mu(CA) = \int_{T(A)} \frac{1}{\det(x)^n} \,dx $$
By Change of Variables Theorem:
$$ \mu(CA) = \int_{A} \frac{1}{\det(Cu)^n} \det(C)^n du = \int_{A} \frac{1}{\det(C)^n \det(u)^n} \det(C)^n du = \int_{A} \frac{1}{\det(u)^n} du = \mu(A) $$
I have two questions:
- Is my approach correct?
- How can I prove that $\mu$ is a regular measure.
First question: You need to be a bit more careful to not mix up elements of $\operatorname{GL}_n(\mathbb{R})_+$ and Borel subsets of $\operatorname{GL}_n(\mathbb{R})_+$. Otherwise your approach is correct (of course you are using $\det(C) > 0$ to omit the modulus) and the right invariance of $\mu$ is shown analogously.
Second question: You first need to check that $\mu$ is a measure that is finite on compact sets (the $\sigma$-additivity of the measure uses dominated convergence, the other properties are pretty clear).
Depending on your definition of regularity you now either need to show inner regularity on every subset, inner regularity on open subsets and outer regularity on every subset, or inner and outer regularity on every subset. Since the group $G := \operatorname{GL}_n(\mathbb{R})_+$ is $\sigma$-compact, these three definitions are actually equivalent here.
Write $U_m := \{M \in G: \frac{1}{m} < \det(M) < m\}$ and note that for any $A \in \mathcal{B}(G)$ we have $$\mu(A) = \lim_{m \to \infty} \mu(A \cap U_m).$$ With the Lebesgue measure $\lambda$ on $\mathbb{R}^{n^2}$ we furthermore have $$\frac{1}{m^n} \lambda(A \cap U_m) = \int_{A \cap U_m} m^{-n} \, dx \leq \int_{A \cap U_m} \det(x)^{-n}\,dx = \mu(A \cap U_m)$$ and $$\mu(A \cap U_m) = \int_{A \cap U_m} \det(x)^{-n} \, dx \leq \int_{A \cap U_m} m^n \, dx = m^n \lambda(A \cap U_m).$$
These two bounds together with the above limit should help you to deduce the necessary regularity properties from the regularity properties of the Lebesgue measure $\lambda$.
As a small remark, this result holds more generally: If $X$ is a locally compact Hausdorff space, $\lambda$ a Radon measure on $X$ and $f: X \to [0, \infty)$ continuous, then $$(f \, d\lambda)(B) := \int_B f \, d\lambda$$ defines a Radon measure on $X$.