Let a and b be natural numbers such that $2a - b, a - 2b$ and $a + b$ are all distinct squares. What is the smallest possible value of $b$?
Let, $2a-b=k^2, a-2b=p^2, a+b=q^2$.
$k^2=p^2+q^2$ after adding any of the two equations.
How to proceed further?
On
The first few quintuples are $$(a,b,a+b,2a-b,a-2b)\\\in\big\{ (6,3,9,9,0 ),\quad (24,12,36,36,0 ),\quad (54,27,81,81,0 ),\quad (96,48,144,144,0 ),\\ \mathbf{(123,21,144,225,81 )},\quad (150,75,225,225,0 ),\quad (216,108,324,324,0 ),\\ (294,147,441,441,0 ),\quad (384,192,576,576,0 ),\quad (486,243,729,729,0 ),\\ \mathbf{(492,84,576,900,324 )},\quad (600,300,900,900,0 ),\quad (726,363,1089,1089,0 ),\\ (864,432,1296,1296,0 ),\quad \mathbf{(939,357,1296,1521,225 )},\quad \cdots\big\} $$ The lowest value of $\space b\space $ that works is either $\space 3\space $ or $\space 21\space $ depending on whether or not you allow $\space (a-2b)=0\space $.
If you subtract the last two you get $q^2-p^2=3b$. If you add the first and last you get $k^2+q^2=3a$. No primitive Pythagorean triangle has legs that differ by a multiple of $3$, so we need a triangle that has a common factor of $3$. The smallest such is $9-12-15$ and we find $$3b=144-81=63\\b=21\\3a=225+144=369\\a=123$$ This is the smallest $b$ because the difference of the two legs must be at least $3$. If the shorter leg is $c$ we have $(c+3)^2-c^2=6c+9$ and $b$ will grow with the shorter leg.