For instance we consider the magic squares of order $3$ with the constant $15$.
We can find : \begin{array}{ | l | c | r | } \hline 8 & 3 & 4 \\ \hline 1 & 5 & 9 \\ \hline 6 & 7 & 2 \\ \hline \end{array} and \begin{array}{ | l | c | r | } \hline -2 & 8 & 9 \\ \hline 16 & 5 & -6 \\ \hline 1 & 2 & 12 \\ \hline \end{array}
Do we have a tool to transform the first matrix into the second ? I mean that does the invariant (same constant) is sufficient to classify all the magic squares of a given constant under a group's action ?
I know that of it was the same numbers for the two matrices, there exists an action from the group $\mathcal{D}_4$ on the set of magic squares of order $3$.
Thanks in advance !
All magic squares of order 3 are based on the same structure. As you note in the question, actions from $\mathcal{D}_4$, the dihedral group of order 8, permute the numbers while preserving the row and column (and diagonal) sums, and hence preserve the magic property.
There are other things we can do which preserve the magic property. For example, we can add a constant to each cell value, or multiply each value by a constant. Thus we can apply any linear operation $x' = ax + b$ to each value.
To see what else preserves the magic property, we need to look at how magic squares can be produced.
Magic squares are closely related to Latin squares. "A Latin square is a $n\times n$ array filled with n different symbols, each occurring exactly once in each row and exactly once in each column". Eg,
$$\begin{array}{|r|r|r|} \hline A & B & C\\ \hline C & A & B\\ \hline B & C & A\\ \hline \end{array}$$
We can create a magic square using a pair of mutually orthogonal Latin squares, that is, two Latin squares of the same order that when superimposed produce distinct pairs of values in each cell. (An old name for this construction is the Graeco-Latin square, but as Wikipedia mentions, that name is rarely used these days, although you may encounter it in works on magic squares).
The order 3 magic square can be produced by combining
$$\begin{array}{|r|r|r|} \hline 2 & 0 & 1\\ \hline 0 & 1 & 2\\ \hline 1 & 2 & 0\\ \hline \end{array}$$
with
$$\begin{array}{|r|r|r|} \hline 1 & 2 & 0\\ \hline 0 & 1 & 2\\ \hline 2 & 0 & 1\\ \hline \end{array}$$
which is just the previous matrix reflected vertically. Note that this square is magic, with a constant sum of 3, although it's not a proper magic square because it has repeated values. When we combine the two squares we get
$$\begin{array}{|r|r|r|} \hline 21 & 02 & 10\\ \hline 00 & 11 & 22\\ \hline 12 & 20 & 01\\ \hline \end{array}$$
The 1st digit in each cell comes from the corresponding position in the 1st Latin square, the 2nd digit comes from the 2nd Latin square. Notice that all of these ordered pairs are distinct.
Now, we treat each of those values as a number written in base 3. This gives us
$$\begin{array}{|r|r|r|} \hline 7 & 2 & 3\\ \hline 0 & 4 & 8\\ \hline 5 & 6 & 1\\ \hline \end{array}$$
which is a valid magic square because there are no repeated values. If we add 1 to each cell, we get
$$\begin{array}{|r|r|r|} \hline 8 & 3 & 4\\ \hline 1 & 5 & 9\\ \hline 6 & 7 & 2\\ \hline \end{array}$$
which is your first matrix.
If we use any linear operation of the form $au + bv + c$ on the $(u,v)$ values in a cell, we'll also get a magic square. We just need to be a little careful when choosing $a$ and $b$ to avoid getting repeated values. For example, with $a=7, b=4, c=-6$ we get
$$\begin{array}{|r|r|r|} \hline 7\cdot2+4\cdot1-6 & 7\cdot0+4\cdot2-6 & 7\cdot1+4\cdot0-6\\ \hline 7\cdot0+4\cdot0-6 & 7\cdot1+4\cdot1-6 & 7\cdot2+4\cdot2-6\\ \hline 7\cdot1+4\cdot2-6 & 7\cdot2+4\cdot0-6 & 7\cdot0+4\cdot1-6\\ \hline \end{array}$$
or
$$\begin{array}{|r|r|r|} \hline 12 & 2 & 1\\ \hline -6 & 5 & 16\\ \hline 9 & 8 & -2\\ \hline \end{array}$$
which is your 2nd matrix, rotated by 180°.
As I said earlier, all magic squares of order 3 are based on the same structure. Clearly, if you permute the rows or columns of a Latin square, the result is still a Latin square. So given any Latin square we can permute it so that its 1st row and column are in natural order, eg
$$\begin{array}{|r|r|r|} \hline 0 & 1 & 2\\ \hline 1 & 2 & 0\\ \hline 2 & 0 & 1\\ \hline \end{array} $$
This is known as the reduced form.
The Wikipedia article on Small Latin squares and quasigroups states that there is only one reduced Latin square of order three. That's pretty easy to see. If we start with
\begin{array}{|r|r|r|} \hline 0 & 1 & 2\\ \hline 1 & & \\ \hline 2 & & \\ \hline \end{array}
there's only one possible way to complete the square.
For a magic square, we need the diagonals to sum to the row constant. So we need to permute the rows & columns of the standard Latin square so that the diagonals of our mutually orthogonal Latin squares are either $(0, 1, 2)$ or $(1,1,1)$. It's easy to see that both diagonals cannot be of the same form, so we need to start with an arrangement like
$$\begin{array}{|r|r|r|} \hline 2 & & 1\\ \hline & 1 & \\ \hline 1 & & 0\\ \hline \end{array}$$
and clearly there's only one way to complete that Latin square:
$$\begin{array}{|r|r|r|} \hline 2 & 0 & 1\\ \hline 0 & 1 & 2\\ \hline 1 & 2 & 0\\ \hline \end{array}$$
which is the first $(0,1,2)$ Latin square given earlier.
We need to choose $a$ and $b$ to ensure that all the cell values are distinct. Our cell value equation is $x = f(u, v) = au + bv + c$. Given two cells, $(u, v)$ and $(u', v')$, we want $f(u, v) \ne f(u', v')$.
Let $$au + bv + c = au' + bv' + c$$ $$a(u-u') = b(v'-v) \tag{1}$$ The possible values of $u-u', v'-v$ are in $\{0, \pm1, \pm2\}$. We only want (1) to be true when $u-u' = v'-v = 0$. So we just need to choose non-zero $a$ and $b$ such that $a \ne \pm b$, $a \ne \pm 2b$, and $2a \ne \pm b$. We are free to choose any $c$.
Here is a Python script that generates order 3 magic squares using the above method, with MathJax output.