Here is the (somewhat lengthy, but necessarily so) set-up. (The context is Gross's paper on Kolyvagin's work, which I am reading for a project. My advisor and I have chatted about this, but I don't see him again until Friday, so I thought I'd check my work here in the meantime.)
Let $K$ be an imaginary quadratic field $\mathbb{Q}[\sqrt{-D}]$, $E$ an elliptic curve defined over $\mathbb{Q}$ of conductor $N$, and $p$ a prime such that the map $\mathrm{Gal}(\mathbb{Q}(E[p])|\mathbb{Q}) \to GL_2(\mathbb{F}_p)$ is an isomorphism. (If $E$ does not have CM, this is true for all but finitely many $p$.) Let $\ell$ be a prime not dividing $N \cdot 2D \cdot p$, so that $\ell$ is unramified in $M = K(E[p])$. Let $\sigma$ be an element of the conjugacy class $\mathrm{Frob}(\ell) \subset (M|\mathbb{Q})$.
What I'm trying to prove: The characteristic polynomial of $\sigma$, as an endomorphism of the $2$-dimensional vector space $E[p]$, is $X^2 - a_\ell X + 1$, where $|\tilde{E}(\mathbb{F}_\ell)| = \ell + 1 - a_\ell$. Below is my attempt, but I think I made a mistake.
$E$ has good reduction at $\ell$ and $p \neq \ell$, so the reduction map $\tilde{\cdot}:E[p] \to \tilde{E}[p]$ (I think this is the mistake) is an isomorphism which induces an isomorphism $\tilde{\cdot}^*:\mathrm{End}(\tilde{E}[p]) \to \mathrm{End}(E[p])$. $\tilde{E}$ is an elliptic curve over the finite field $\mathbb{F}_\ell$, so the Frobenius map $F:(x,y) \mapsto (x^\ell, y^\ell)$ acts on $\tilde{E}$. $F$ defines an endomorphism on the $2$-dimensional $\mathbb{F}_p$ vector space $\tilde{E}[p]$, and its characteristic polynomial acting on $\tilde{E}[p]$ is $$X^2 - a_\ell X + \ell.$$ I understand very well why this fact is true, e.g. it's in Silverman's book, Ch. V Theorem 2.3.1, or see the related Question about characteristic polynomial of the Frobenius endomorphism on elliptic curves.. This is a different question from mine: You understand the mod-$p$ Frobenius action on the $p$-torsion of an elliptic curve defined over $\mathbb{F}_\ell$; I am having trouble passing from this to the $p$-torsion over the number field.
Recall the Frobenius element $\sigma$, so $\sigma$ acts as $\alpha \mapsto \alpha^\ell$ on the extension of residue fields $\mathbb{Z}/\ell \subset \mathcal{O}_M/\mathfrak{L}$. By the definition of a Frobenius element $\sigma$, $\tilde{\cdot}^*$ maps $F$ to $\sigma$, and maps $0 = F^2 - a_\ell F + \ell \to \sigma^2 - a_\ell \sigma + \ell$, so $$\sigma^2 - a_\ell \sigma + \ell = 0.$$ Therefore $\sigma$, acting on $E[p]$, has characteristic polynomial $X^2 - a_\ell X + \ell$.
However, this is not correct because there are many elements of $\mathrm{Gal}(M|\mathbb{Q})$, not just $\sigma$, to which $F$ could map-- the extension is non-abelian, so Frobenius is a whole nontrivial conjugacy class. I expect the mistake lies in the assertion about $\tilde{\cdot}^*$ being an isomorphism. However, characteristic polynomial is a conjugacy class invariant (since $P(BAB^{-1}) = BP(A)B^{-1}$ and the dimension is $2$) so there has got to be a way to move through this logic without insisting $F \mapsto \sigma$. One thing that might help is to use the Weil pairing, which I think can be used to show the action of $\sigma$ is diagonalizable on $E[p]$.
Thanks to anyone who answers for the help!
From @Mathmo123's comment (thank you!), I think there is an easy fix. I'm answering my own question in case it helps others. It is useful to actually write down the prime of $M$ whose Frobenius $\sigma$ is. If $\mathfrak{L}$ is a prime of $M$ lying over $\ell$, we have the decomposition group $$D_{\mathfrak{L}|\ell} = \{\sigma \in \mathrm{Gal}(M|\mathbb{Q}):\sigma \mathfrak{L} = \mathfrak{L}\},$$ and an isomorphism $$D_{\mathfrak{L}|\ell} / I_{\mathfrak{L}|\ell} \to \mathrm{Gal}(\mathbb{F}_\mathfrak{L}|\mathbb{F}_\ell).$$ $\ell$ is unramified so inertia is trivial, and $D_{\mathfrak{L}|\ell}$ is cyclic of order $f$. Changing the particular element of the conjugacy class $\mathrm{Frob}(\ell)$, say conjugating by $\mu$, changes $D_{\mathfrak{L}|\ell}$ to the subgroup $\mu D_{\mathfrak{L}|\ell} \mu^{-1}$, which is the decomposition group of the prime $\mu\mathfrak{L}$.
Now the map $\tilde{\cdot}^*$ is an isomorphism of the Galois modules $\mathbb{F}_p [D_{\mathfrak{L}|\ell}]$ and $\mathbb{F}_p [\mathrm{Gal}(\mathbb{F}_\mathfrak{L} | \mathbb{F}_\ell)]$ acting on $E[p]$ and $\tilde{E}[p]$ respectively, and the rest of the argument goes through.
I was trying to have this to be an isomorphism with $\mathrm{Gal}(M|\mathbb{Q})$ instead of $D_{\mathfrak{L}|\ell}$, which was silly. Indeed, if some element of $\mathrm{Gal}(M|\mathbb{Q})$ does not lie in any of the decomposition groups (there is such an element because all the decomposition groups have at most $g(f-1) + 1 < fg = |\mathrm{Gal}(M|\mathbb{Q})|$ distinct elements amongst them), the map wouldn't even make sense there. Instead there's as many conjugate decomposition groups as primes lying over $\ell$, each mapping isomorphically to $\mathrm{Gal}(\mathbb{F}_\mathfrak{L}|\mathbb{F}_\ell)$. Not writing down the dependency on which $\mathfrak{L}$ was muddying the situation.