I want find the characteristic polynomial for the grover quantum operator $U^{N\times N}$
$$\begin{align*} U=(2|D\rangle\langle D| -I_N)(2|M^{\perp}\rangle\langle M^{\perp}| -I) \end{align*}$$
where $|D\rangle = \dfrac{1}{\sqrt{N}}\sum_{i=1}^{N} |i\rangle$; $|M\rangle = \dfrac{1}{\sqrt{N-m}}\sum_{i\notin M} |i\rangle$ ; $N=2^n$
I will assume $|M^{\perp}\rangle$ is just a normalized vector. Notice that the subspace $\mathcal{S}$ generated by $|D\rangle$ and $|M^{\perp}\rangle$ is an invariant subspace for $U$. Let me define $|e_1\rangle =|D\rangle$ and $|e_2\rangle = |M^{\perp}\rangle$. Expanding the product you will notice that
$$ U |e_i\rangle = \sum_{j=1}^{2} U_{j,i} |e_j\rangle $$
and the coefficients $U_{i,j}$ are essentially functions of $\langle D|M^{\perp}\rangle$. In this way you obtained a matrix representation of your operator $U$ in $\mathcal{S}$. This is a $2\times2$ matrix and you should be able to find the eigenvalues easily. On the remaining subspace $\mathcal{S}^\perp$ It should be clear that
$$U |\xi\rangle = |\xi\rangle, \quad \mathrm{ for } \quad |\xi\rangle \in \mathcal{S}^\perp$$
So in this subspace you find eigenvalue 1 with multiplicity $N-2$ (its dimension). Now you have all the eigenvalues.
Note that your vectors $|D\rangle$ and $|M^{\perp}\rangle$ don't need to (and in general won't) be orthogonal. But the determinant is defined irrespective of the scalar product and is hence invariant under any change of basis (read similarity transformation). However to realize that the above procedure works it may be helpful to imagine that you could in principle find an orthonormal basis whose first two vector (say) span $\mathcal{S}$ and the remaining span $\mathcal{S}^\perp$.
Hint
In the subspace $\mathcal{S}$, in the basis $|D\rangle, |M^{\perp}\rangle$, $U$ has the form (if I'm not wrong)
$$ \left. U \right|_{\mathcal{S}}= \left(\begin{array}{cc} 4\left|x\right|^{2}-1 & 2\overline{x}\\ -2x & -1 \end{array}\right) $$
with $x=\langle D|M^{\perp}\rangle $. Correctly its eigenvalues have modulus one. In fact if $P$ is an orthogonal projector, $2P-I$ is unitary so that your $U$ is unitary being a product of two unitary.