For some arbitrary configuration of a neural network, say a basic multi layer perceptron, is there a way to break down a network into a number of smaller networks, compute them individually, and recombine them?
For matrix multiplication, you can do something like this, where if you have two 16x16 matricies, you can break them into 8x8 blocks and multiply the individual blocks together, then add them together to get the same result as doing the 16x16 multiplication, without doing any unnecessary computation.
I am wondering if there is any kind of similar mechanism someone could use if they had some kind of hardware multiplier capable of computing a network of a fixed size and wanted to use it to compute a larger network.
Thanks for any help you can offer!