The connection between Image Segmentation with Neural Networks versus graph cut and mumford-shah type approaches

268 Views Asked by At

This question is meant to be a more technical question, so if I have phrased it as soliciting an opinion, any suggested language updates would be appreciated.

I am relatively new to image processing, but have been looking at both the Convolutional Neural Networks literature on image segmentation, as well as the literature on graph cut and Mumford-Shah type approaches to image segmentation. I understand that graph cuts is a particular interpretation of the image segmentation problem, and that the Mumford-Shah functional is one implementation of penalty by which to identify the cut edges. I have yet to find a Convolutional Neural Network paper that references the Mumford-Shah approaches, and vice versa.

Can anyone explain where Mumford-Shah approaches work better than CNNs, and where CNNs work better than Mumford-Shah. Are there particular domains in which one approach works better compared to the other--such as noisier backgrounds, etc. I realize that these questions are still open research areas.

Graph cut ideas of course extend to problems outside of image processing, however I was wondering if anyone has directly compared the efficiency or prediction error of graph cut image segmentation versus CNNs? CNNs face a number of challenges in generalizing to problems outside of their training sets, but I have not seen how well graph-cut methods compare on the same datasets.

It seems like the two literatures are proceeding in parallel, though of course there are obvious overlaps.

Any suggestions would be appreciated.

2

There are 2 best solutions below

5
On

You can't really compare graph cuts to CNNs. We use graph cuts to solve flow problems, and we use CNNs to learn patterns. We would only see this type of comparison if one was using each approach to solve a very specific problem (even more specific than image segmentation). While each method can be applied to segmentation, they have very different purposes and are applied to different problems.

1
On

GraphCuts performs much better than CNNs if the pixel labels are scarce. CNNs only perform better than GraphCuts if most pixels are labels. So you see, GraphCuts only needs weak supervision, but CNN needs full supervision, this is the biggest difference. Also, CNNs (and neural networks in general) have been facing the problem of lacking labelled data, and researchers investigated a lot into how to use GraphCuts to generate more labelled data to train CNNs, for example.