Consider the following matrices over $\mathbb{F}_2$
$$A = \left[\begin{matrix} 1 & 1 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}\right]\ ,\ x = \left[\begin{matrix} x_1 \\ x_2 \\ x_3 \end{matrix}\right]\ ,\ b = \left[\begin{matrix} 1 \\ 0 \\ 1 \\ 0 \end{matrix}\right]$$
I'm trying to find the vector $x$ that minimizes the hamming distance between $Ax$ and $b$. From doing some reading online, I've discovered that the assignment $x$ which minimizes $||Ax-b||^2$ (pretty much equivalent to the hamming distance) is given by $x=A^+b$, where $A^+$ is the Moore-Penrose pseudo-inverse, a matrix which satisfies $AA^+A = A$ (and a few other conditions). For the above example, this matrix is
$$A^+ = \left[\begin{matrix} 1 & 0 & 1 & 1 \\ 0 & 0 & 1 & 0 \\ 1 & 1 & 0 & 0\end{matrix}\right] \implies x=A^+b = \left[\begin{matrix} 0 \\ 1 \\ 1 \end{matrix}\right]$$
However, $Ax = \left[\begin{matrix} 0 & 1 & 1 & 1 \end{matrix}\right]^T$, which has hamming distance 3 from $b$. You can trivially find other solutions through brute force (such as $x=\left[\begin{matrix} 1 & 1 & 1 \end{matrix}\right]^T \implies Ax=\left[\begin{matrix} 1 & 0 & 1 & 1 \end{matrix}\right]^T$) which only have hamming distance 1 from $b$.
What's going wrong, and how can I get least squares to work in $\mathbb{F}_2$?