I'm solving a math puzzle: "how quickly can you multiply $26$ by $24?$"
I don't know the answer so I use tutorials.
One tutorial say to do it quickly you can round numbers up and down to closest power of $5$ or $10$.
In my example $26\times 24$ I round $26$ down by $1\to 25$, and I round $24$ up by $1\to 25$. I added and removed $1$ from both number so the sum of my change is 0.
Now I have two easy number $25\times 25$ which is $625$.
But now if I use calculator program and multiply $24\times 26$ I get $624$.
Where did 1 go? What did I do wrong?

Rounding can get us a close answer, but it might not be the same. For example, what is $4\times 6?$ What is $5\times 5?$
In your case, $26\times 24$ is the total of $26$ separate groups of $24$ members each, while $25\times 25$ is the total of $25$ separate groups of $25$ members each. Note that $26\times 25$ has $26$ more than $26\times 24,$ since each of the $26$ groups has exactly one more member. That is: $$26\times 25=26\times 24+26.$$ On the other hand, $25\times 25$ has $25$ fewer members than $26\times 25,$ since it has one fewer group (of $25$). That is: $$25\times 25=26\times 25-25.$$
Consequently, we see that $$25\times 25=26\times 25-25=(26\times 24+26)-25=26\times 24+26-25=26\times 24+1,$$ as you've already discovered.
Put geometrically, you've stumbled upon the fact that a square has a greater area than a non-square rectangle having the same perimeter. Let's see why that is! I'll describe how to picture it, and I suggest that you actually draw pictures.
Start with a rectangle that is $a$ by $b,$ with $b$ bigger than $a.$ Now, draw a line cutting the rectangle into a square (that is $a$ by $a$) and a small rectangle (that is $b-a$ by $a$). Now, cut the small rectangle into two smaller rectangle that are both $\frac12(b-a)$ by $a.$ Move one of the smaller so that the smaller rectangles have an edge coinciding with neighboring edges of the square. Now, if we include a $\frac12(b-a)$ by $\frac12(b-a)$ square touching both of the smaller rectangles, then we have increased the area, and we have created a square whose perimeter is the same as the rectangle we started with.