VC Dimension for Non Linear Models

Question

For non linear models like neural networks the number of VC dimensions could be larger then the number of parameters. Is this a correct statement , I tried looking for leads but was not able to find any good ones. Any pointers or explanation would be very helpful.

Answer 1

The VC dimension of shallow neural networks grows roughly as $WlogW$ whereas it is $W^2$ for deep neural networks, where $W$ is the number of parameters in the network. The main reference on this topic is the book "neural network learning: theoretical foundations" by Anthony and Bartlett.