How to arrange these detected points from one end points to other, edge to edge of an object?

Question

Please suggest what topic to tag this question if I have tagged it in the wrong topic, after going through this.

I have detected partial edge of an object like this after image processing.

I have reduced the edges into discrete point like this. The inbuilt function detects these edges like the [x y] coordinates as given below, but unfortunately sometimes it doesn't start and end at the end points of the edge those are detected.

I would like to arrange these points in order from one end point to other end point.

I would like to remind that while dealing with images the Origin is on the left top corner, unlike right bottom corner on a normal graph. So the consequence is that X coordinates increase from left of the image to right and Y coordinates increase from top to bottom

Give you an idea

Origin         X values increase----->
(0,0)


Y values increase
|
|
⬇

The Coordinates detected

[[[ 639 1066]]
 [[ 633 1068]]
 [[ 627 1069]]
 [[ 621 1073]]
 [[ 615 1077]]
 [[ 609 1082]]
 .....
 [[ 679 1087]]
 [[ 672 1081]]
 [[ 665 1075]]
 [[ 655 1069]]]

Is there any logic which I can use to rearrange these points from one end point to the other?