All square roots can be represented as a continued fraction.
The fraction can be calculated to $n$ terms (e.g. $\sqrt{2}$ is $[1; 2, 2, 2, 2...]$)
So the continued fraction for $\sqrt{2}$ to $3$ terms would be $$1 + \frac{1}{2 + \frac{1}{2 + \frac{1}{2 + 1}}} = \frac{17}{12}.$$
Additionally, the next convergent for square root of $2$ can be calculated with the formula $$\frac{numerator + 2 * denominator}{numerator + denominator}$$
How would I find the simplified continued fraction to $Y$ terms for the square root of $N$? So $f(2, 3)$ would be $\frac{17}{12}$.
If a similar formula to calculating the next convergent of $\sqrt{2}$ exists for all square roots, please include that formula.
I think you can construct something form
$$\sqrt{a}=1+\sqrt{a}-1=1+\frac{(\sqrt{a}-1)(\sqrt{a}+1)}{1\cdot(\sqrt{a}+1)}$$ $$=1+\frac{a-1}{1+\sqrt{a}}$$
This equation is self referential. So if you plug in the the expression on the right hand side of the equation for $\sqrt{a}$ in the denominator you get. $$\sqrt{a}=1+\frac{a-1}{1+[1+\frac{a-1}{1+\sqrt{a}}]}=1+\frac{a-1}{2+\frac{a-1}{1+\sqrt{a}}}$$
Now you can repeat this as long as you wish. I hope that was what you were asking for.
And there is a way of directly getting the convergents. Euler did this in a paper.