I was trying to find $W,b$ such that $f(x) = W^T x + b = 0$ and that there exists more than one point in a hypercube $x \in [0,4]^D$ that satisfy $f(x) = 0$. In other words, I have a hypercube and I want to find a plane that goes through/cuts the hypercube but at the same time, those points that cut the plane must satisfy $f(x) = 0$. Ideally, we cut the hypercube not right at the edge, so that we actually cut through a whole piece of the hypercube.
To solve this and understand the problem in a simpler scenario I scaled it down to $D = 2$.
In this case when the hyperplane $f(x) = 0$ means that we have $W_1 x_1 + W_2 x_2 + b = 0$. In this case to find parameters $W,b$ to make that true I choose 3 points $a_1, a_2, a_3 \in [0,4]^2 $ such that $f(x) = 0$. Intuitively I choose 3 because I have three unknowns so I wanted to have enough equations and unknowns. I chose the points $a_1 = [2,2], a_2 = [4,0], a_3 = [0,4]$. With this I got the linear system:
$$ A[w,b] = \left( \begin{array}{ccc} 2 & 2 & 1 \\ 4 & 0 & 1 \\ 0 & 4 & 1 \end{array} \right)\left( \begin{array}{ccc} W_1 \\ W_2 \\ b \end{array} \right)=\left( \begin{array}{ccc} 0 \\ 0 \\ 0 \end{array} \right) $$
to my (no surprise) since the matrix A is invertible I found that the only solution that exists is $W = [0,0], b = 0$. Which from a linear algebra perspective with the system I set up makes sense. However, what does not make sense to me is the geometric intuition of the solution. What I was actually trying to attempt to do is choose plane that goes through the middle of the hypercube and that it also goes through two corners of the hypercube that I chose. This is what my picture looks like:
my intuition tells me that indeed there should be a unique plane that crosses those points (in this case a line but in reality we have a plane because we are choosing $f(x) = W^Tx + b$ to lie inside the hypercube) but the plane shouldn't the trivial silly equation $f(x) = 0x_1 + 0x_2 + 0$, which isn't even a plane and essentially is equal to zero everywhere. Where did I go wrong? Why wasn't I able to choose the hyperplane I intended to choose?
"...the matrix A is invertible.." is false: $\det A = 0$ and you have nontrivial solutions, namely $W_1 = W_2 = -b/4\ne 0$.