Simple question: If I want to fit data points to a function of the form $y=a+\frac{b}{x}$, is there any reason why I can't use a least-squares approach?
I want to minimize $$E=\sum_{i=1}^{n}(y-a-\frac{b}{x})^2$$
So just set $\frac{\partial E}{\partial a}=\frac{\partial E}{\partial b}=0$ and solve the linear system?
Problem statement
Start with a set of $m$ measurements $\left\{ x_{k}, y_{k} \right\}$ and the model function $$ y(x) = a + \frac{b}{x}. $$ Assume the data are distinct to the point where the system matrix will have full column rank. Assume none of the $x$ values are $0$.
Least squares definition
The least squares solution is defined as $$ (a,b)_{LS} = \left\{ (a,b)\in \mathbb{R}^{2} \colon \sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2} \text{ is minimized} \right\} $$
Linear system
$$ \begin{align} \mathbf{A} a &= y \\ % \left[ \begin{array}{cc} 1 & \frac{1}{x_{1}} \\ \vdots & \vdots \\ 1 & \frac{1}{x_{m}} \\ \end{array} \right] % \left[ \begin{array}{c} a \\ b \\ \end{array} \right] &= % \left[ \begin{array}{cc} y_{1} \\ \vdots \\ y_{m} \\ \end{array} \right] % \end{align} $$
Linear system solution
The least squares solution for the full column rank problem is $$ % \left[ \begin{array}{c} a \\ b \\ \end{array} \right]_{LS} = \mathbf{A}^{+} y $$
One path to this solution is to use the normal equations: $$ \begin{align} \mathbf{A}^{*} \mathbf{A} \, a &= \mathbf{A}^{*}y \\ % \left[ \begin{array}{cc} \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot \frac{1}{x} \\ \frac{1}{x} \cdot\mathbf{1} & \frac{1}{x} \cdot \frac{1}{x} \\ \end{array} \right] % \left[ \begin{array}{c} a \\ b \\ \end{array} \right] &= % \left[ \begin{array}{cc} \mathbf{1} \cdot y \\ \frac{1}{x} \cdot y \end{array} \right] % \end{align} $$ These are exactly the same equations you would get by the minimization process you outlined: $$ \begin{align} % & \frac{\partial}{\partial a} \sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2} = 0, \\ % & \frac{\partial}{\partial b} \sum_{k=1}^{m} \left( y_{k} - a - \frac{b}{x_{k}} \right)^{2} = 0. \\ % % % \end{align} $$
The solution is then $$ \left[ \begin{array}{c} a \\ b \\ \end{array} \right]_{LS} = \left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} y $$
(These remarks are an elaboration of the points made by @David.)