Consider the (geometrically reducible) modular curve $Y(N)$ over $\mathbb{Q}$, representing the functor
$$Y(N)(T)=\{(E/T,\alpha),\; E/T \; \text{elliptic curve}, \;\alpha:(\mathbb{Z}/N\mathbb{Z})_T^2\overset{\sim}{\to}E[N] \}/\sim $$
for $\mathbb{Q}$-schemes $T$. How does the absolute Galois group $\mathrm{Gal}_\mathbb{Q}$ act on $Y(N)_\overline{\mathbb{Q}}$?
It should be something like $\sigma \in \mathrm{Gal}_\overline{\mathbb{Q}}$ mapping a pair $(E,\alpha)$ to something like $(E^\sigma,i_\sigma \circ\alpha)$, where $E^\sigma$ is the elliptic curve with $\sigma$ applied to the equations defining $E$ and $i_\sigma$ is the isomorphism $E\to E^\sigma$.
However aren't two pairs $(E,\alpha),(E',\alpha')$ equivalent if there is an isomorphism $f:E\to E'$, such that $\alpha'\circ f=\alpha$? By this logic we would have $\sigma.(E,\alpha)=(E,\alpha)$, but I'm pretty sure I'm making a mistake somewhere.
As I understand a lot of (and sometimes all?) automorphisms of $Y(N)_\overline{\mathbb{Q}}$ come from the natural action of $\mathrm{GL}_2(\mathbb{Z}/N\mathbb{Z})$ on $Y(N)_\overline{\mathbb{Q}}$. Does the absolute Galois group also act in terms of this matrix action?
You are almost right, but you are forgetting the base ring.
For a $\mathbb{Q}$-algebra (or $\mathbb{Z}[1/N]$-algebra) $R$, $Y(N)(R)$ is the pairs $(E, \alpha)$ defined over $R$, modulo isomorphisms of $R$-schemes $f : E \to E'$ compatible with $\alpha, \alpha'$.
If you take a random element of $Gal_{\mathbb{Q}}$ and act on $(E, \sigma)$ by it, then $\sigma$ certainly gives a map $(E, \alpha) \to (E^\sigma, \alpha^{\sigma})$ but it is not the identity on the coefficient ring $\bar{\mathbb{Q}}$ (it is $\sigma$, of course) so it does not show that they are the same point of $Y(N)$.