CNN AlexNet algorithm complexity

Question

I'm first year student in machine learning and I really recently started to immersing.

So, professor gave me a task, calculate number of:

• matrix additions
• matrix multiplications
• matrix divisions

Which are processed in the well known convolutional neural network - AlexNet.

I found some matherials about it, but I really confused where to start.

So, the overall structure might looks like:

But, how can I calculate operations for each type distinctly?

Answer 1

First, you have to make a decision: Do you want to use the "real" alexnet (with the grouping) or what most frameworks use as AlexNet (without grouping).

In case you choose without grouping, you might want to have a look at Table D2 of my masters thesis for a better overview over the layers. Especially the output size / number of filters / stride. Don't take the number of FLOPs too seriously, it is rather a ballpark-estimate.

Then you have to ask you the following:

• How would you implement convolution?
• How would you implement max pooing?
• How would you implement a fully connected layer?

I will not give you the answer directly here, but recommend to have a look at the implementation of a framework of your choice. Or you could search for "pure numpy cnn implementation" or something similar, e.g.

https://github.com/llSourcell/Convolutional_neural_network/blob/master/app/model/alpha_cnn_predict.py#L72-L98