Currently, neural networks can be trained in a parallel fashion and I read a couple of very good research papers on it. I am trying to implement neural networks that can take concurrent input and compute the output.
For example, rather than directly taking an input vector $\\v=\begin{bmatrix} x_{1}\\ x_{2}\\ \vdots\\ x_{n} \end{bmatrix}\\$ and activating the neural network to produce certain output $\\y\\$, it should take a series of binary vectors that add up to the main vector. For example let there be a normal neural network $\\f\\$ and a special kind of neural network $\\f_o\\$ with initial state $\\X_o\\$, an input vector $\\I=\begin{bmatrix} 1\\ 3\\ 2 \end{bmatrix}\\$, rather than providing this as an input, neural network should take binary vectors like $\\\bar I= \{ \begin{bmatrix} 0\\ 1\\ 0 \end{bmatrix},\begin{bmatrix} 0\\ 1\\ 1 \end{bmatrix},\begin{bmatrix} 1\\ 1\\ 1 \end{bmatrix} \} \\$ such that $\\I = \bar I_1+\bar I_2+\bar I_3\\$.
Rather than directly passing the decimal input vector to the neural network
$\\f(I) = y\\$
I want to implement $\\f_o(\bar I_1)(\bar I_2)(\bar I_3) = y\\$. With every input vector, the state of function changes from $\\X_i \to X_{i+1}\\$ such that
- $\\f_o\\$ allows orderless currying i.e. $\\f_o(\bar I_1)(\bar I_2)(\bar I_3) = f_o(\bar I_1)(\bar I_3)(\bar I_2) = f_o(\bar I_3)(\bar I_2)(\bar I_1) = ...\\$
- The only thing that changes the state of neural network/machine is "1" values of the input vector i.e. $\\f_o([1]) : X_i \to X_{i+1}\\$ and $\\f_o([0]) : X_i \to X_{i}\\$
- And statement 2 said, any integer vector $\\v\\$ could be decomposed in vectors $\\\geq \max(v)\\$
How can this be implemented? Should I develop a FSM and Neural Network hybrid? Are there current alternatives for the same? I am afraid if I know all the important keywords, please feel free to edit the tags
Kindly provide me some direction for the same.
I have completed significant parts of the project but this area is something I am currently working on. I hope the community could provide me support. Thanks in advance