I'm currently implementing an algorithm to split an image into smaller chunks, based on straight line separators.
Here's the image I'm processing. It's very small, so you may want to save it and enlarge.

If you look at this image, you can clearly see 3 different zones. But because I used JPG compression, the separator lines have variable pixel colours.
With the adequate colour difference threshold, it's easy to find these separators. Here below are 10 identified separators, or more specifically the average RGB colour of the pixels they contain.
# the three big horizontal separators
row 1 : (64, 43, 32)
row 5 : (60, 46, 29)
row 10: (53, 46, 46)
# now if you use these horizontal lines to split the image,
# you still have two images (do not include the horiz. separators)
# the top one contains three vertical separators
col 1 : (54, 45, 42)
col 8 : (152, 124, 81)
col 10: (43, 49, 43)
# and the bottom contains four
col 1 : (53, 48, 36)
col 5 : (53, 50, 30)
col 6 : (52, 46, 32)
col 10: (43, 52, 45)
As we can see, one of these color average stands out. And as you may see on the picture, it's not really a separator, but only a column of three pixels alike.
I need to discard this fake separator from the set, but I'm having a rough time figuring out an algorithm for this. Initially I'd base this on a standard deviation, removing the ones that are too far away. I don't want to do an average here, as there could be few separators in some cases, and it may lead to false positives.
But I'm lacking mathematical education here. How can I do that on vectors, can anyone provide a working (or better) solution?