Examples of Matrices with "Easy" Inverse and "Hard" Determinant.

Question

A $n\times n$ matrix $M$ is lower triangular if it takes the form $$M=\begin{pmatrix}m_{11} & 0 & ... & 0 \\ m_{21} & m_{22} & ... & 0 \\ \vdots & \vdots & & \vdots \\ m_{n1} & m_{n2} & ... & m_{nn} \end{pmatrix}$$ The determinant of a lower triangular matrix is given $\det(A)=\prod_i M_{ii}$, which can be computed with just $n$ multiplications. Computing the inverse $M^{-1}$ with Gaussian Elimination requires $O(n^3)$ operations. Triangular matrices thus seem to constitute a type of matrix where we can compute determinants faster than inverses. At least if we limited ourselves to currently published algorithms.

Question: I'm interested in matrices where the opposite holds. That is, matrices for which it is easier to compute inverses than determinants. I've listed a few examples below, but they all have determinant $-1$ or $+1$. I'm curious to find more examples, preferably, examples of matrices with less restriction on the determinant.

Known Examples

1. Orthogonal Matrices: A square matrix $Q$ is orthogonal if $Q^T=Q^{-1}$. In this case it is easy to compute the inverse, we just need to take the transpose. The transpose can be computed in $O(1)$ operations as done in e.g. NumPy. The determinant of orthogonal matrices is either $-1$ or $+1$.

2. Involutory matrix: A square matrix $A$ is an involution if $A=A^{-1}$. The inverse is trivial in this case. The determinant of involutory matrices is again either $-1$ or $+1$.

3. Householder Matrix: Let $H=I - 2vv^T$ for a unit vector $v$. It can be shown that $H=H^T=H^{-1}$. These matrices are unitary and involutory so they also have determinant $-1$ or $+1$.

Known NON-Examples

1. Vandermonde Matrix: Please see link for definition. Several algorithms compute inversion in $O(n^2)$ time. There is a determinant formula which allows us to compute the determinant in $O(n^2)$ time.

2. Circular Matrices: Please see link for definition. These matrices are diagonalizable by the Discrete Fourier Transform, so both determinant and inversion can be found in $O(n \cdot \log n)$ time.

Useful Realizations

1. The matrices from the 'known examples' could be scaled with a constant $c\in \mathbb{R} $ and obtain determinant $-c$ or $+c$, so it might be interesting to find a matrix where the determinant is in an interval, e.g. $\det(M)\in [-1, 1]$.
2. The matrix inverse can't be computed with Gaussian Elimination or Diagonalization, as both methods also allow us to compute the determinant along with the inverse. We thus need types of matrices where there are specific algorithms for matrix inversion.