I defined a very simple neural network of $2$ inputs, $1$ hidden layer with $2$ nodes, and one output node. For each input pattern $x⃗ ∈ ℝ×ℝ$ and associated output $o∈ℝ$, the resulting nonlinear equation is:
$wo_{0} σ(x_0 Wi_{00} + x_1 Wi_{10}) + wo_{1} σ(x_0 Wi_{01} + x_1 Wi_{11}) = o$
where $Wi$ is the weight matrix of order $2×2$, where each element $Wi_{jk} \in ℝ$, of input connections, $σ(x)=\frac{1}{1+exp(−x)}$, and $\vec{wo}$, with $wo_{i} \in ℝ$, is the weight vector of the two output connections before the output node.
Given a dataset of $n$ (pattern, output) examples, there will be $n$ nonlinear equations.
I'm asking how to find the solutions of those nonlinear systems, as an alternative method to solve the learning problem, without backpropagation. I've implemented an optimizer for the stated probem. If someone is interested I can provide the relative C sources (email: [email protected]).