Ideal aggregation function for Partially Connected Neural Network (PCNN)

Question

I am building a Python library that creates Partially Connected Neural Networks based on input and output data (X,Y). The basic gist is that the network graph is arbitrarily updated with nodes and edges in the hidden layers.

Example Data:

X = np.array([[0,0],[0,1],[1,0],[1,1]], dtype=np.float)
Y = np.array([[0],[1],[1],[0]], dtype=np.float)

Example Graph:

I am currently using the sum product aggregation function to calculate each layer's values:

$$ \sum_{i=0}^n w_{ij}x_i $$

The synapse weights are denoted by two subscripts $ij$. Subscript $i$ represents the previous neuron and subscript $j$ represents the current neuron under consideration. The sum product for current neuron $j$ is computed as:

$$ s_j = w_{0j}x_0 + w_{1j}x_1 + ... + w_{nj}x_n = \sum_{i=0}^n w_{ij}x_i $$

Is sum product an ideal aggregation function for PCNNs? Dot product won't work as the tensor sizes are almost always incompatible.

** I am self-taught so please forgive any glaring linear algebra misses.

Answer 1

My solution was to use a weight vector $w$ that is padded with zeros and reshaped for dot product. The community support for dot product is just so much better than other methods like Kronecker Product! Here is how I accomplished it in my Python library:

from functools import reduce
import numpy as np
import math

# An example of some input/sensor data
X = np.array([[0,0],[0,1],[1,0],[1,1]], dtype=np.float16)
# From the graph, two inbound synapses in layer 1, here randomly generated
W = np.random.random((2,))

# Enumerate the element count in each array
xElements = reduce(lambda x, y: x*y, list(X.shape))
wElements = reduce(lambda x, y: x*y, list(W.shape))

# If previous layer has more elements than the weights in the current layer
if xElements > wElements:
    # Create a list of zeros of shape X.shape
    A = np.zeros(X.shape).flatten().tolist()
else:
    # Otherwise, create a list with the larger number of elements in W
    A = np.zeros(W.shape).flatten().tolist()

# If the list is odd, add one element to ensure it is even
# Done so the array can be reshaped to fit the data and dot product
if len(A) % 2 != 0:
    A.extend([np.float16(0.0)])

# Fill the list with the elements from W
for ix,ele in enumerate(W):
    A[ix] = ele
A = np.array(A)

# Convert list to an array and reshape to be compatible with dot product
A = A.reshape(X.shape[-1],(math.floor(A.shape[0]/2)))

result = np.dot(X,A)

The result of the dot product looks like:

print(result)
array([[0.        , 0.        , 0.        , 0.        ],
       [0.        , 0.        , 0.        , 0.        ],
       [0.45510922, 0.79058734, 0.        , 0.        ],
       [0.45510922, 0.79058734, 0.        , 0.        ]])