I want to know what is the math behind this tool,
the 10 point divider (Full size image here).
This tool is used to measure equidistant spaces and it's proportional, so you can scale it as much as you want. See this page for its use drawing perspective grids.
I searched the internet but didn't find anything useful except for a golden ratio ruler.
I know that it measures 1 to 1.618 but how to keep it scalable?
I have never seen this device, but I would recognize the geometric principal anywhere. This tool relies on parallelograms to keep its fingers straight. Believe it or not, most of this device is mathematically redundant. To fully constrain the fingers, you only need the bottom three rows of joints.
Thanks to the properties of parallelograms, we can postulate that all of the parallelograms in a row are congruent, because all of their corresponding segments are parallel and of equal length (the distance between two joints). Because any two fingers are just extensions of the opposite sides of a parallelogram, they must be parallel and equally spaced.
As for the Golden Section Gauge, the same principal applies. In the image that you provided, $ \overline{AF}/\overline{BF} = 340/210 \approx \phi $. Parallelogram $ABEC$ keeps $ \overline{AC} \ || \ \overline{BE} $. The Proportional Segments Theorem proves that $ \overline{AF}/\overline{BF} = \overline{FG}/\overline{HG} $, thus $ \overline{FG}/\overline{HG} \approx \phi $. The device is "scaleable", because this proof holds true regardless of the size of $ \angle{FAH} $.