Recently I wondered if there is a neural network topology that can be flexible and adapt to new kinds of data and able to be trained continuously.I found a good paper on Cascade Neural Networks and I think this topology overcomes many of the problems that traditional NN's suffer such as choosing right number of layers or hidden units.
Reading the paper it sounds as if this is the perfect architecture - quick learning , high accuracy , adaptive to changes.How come this topology isn't as widely spread as deep networks, which are so hard to fine tune and train?
I can't find any real world problems that were solved with Cascade NN.What are the cons of using this architecture ?
One disadvantage is that it is much harder to implement than a standard multilayer Perceptron. Another disadvantage is that this is for "standard" feed forward networks, but not for CNNs / RNNs.
One architecture which is closely related to the cascade part and is for CNNs are the recently developed Dense Nets: https://arxiv.org/abs/1608.06993
I am pretty sure DenseNets will be wide-spread quite soon.