Suppose I have a nonlinear least squares objective function I want to minimize: $$ \chi^2(\mathbf{x}) = \sum_{i=1}^n f_i(\mathbf{x})^2 = \mathbf{f}(\mathbf{x})^T \mathbf{f}(\mathbf{x}) $$ Now suppose the solution is not unique, so that $\mathbf{x}_1$ and $\mathbf{x}_2$ both represent global minimum values of $\chi^2$.
I would like to get a sense for ``how big'' is the solution space (i.e. how much non-uniqueness is there in the objective function?). For example, are there a finite number of solutions or an infinite number?
Is there some way to answer these questions by analyzing the Jacobian of the objective function, $$ J = \frac{\partial \mathbf{f}}{\partial \mathbf{x}} $$ For example, would the size of the nullspace of the Jacobian provide any information on the size of the solution space for the $\mathbf{x}$ values? Or is it possible that the Jacobian can be full rank and still have a non-unique solution?
Alternatively, suppose I assume that $\mathbf{x}$ is a solution, and I manage to find a class of matrices $A$ such that $$ \chi^2(A \mathbf{x}) = \chi^2(\mathbf{x}) $$ Certainly if I could find all such matrices $A$, then I would know not only the dimension of the solution space, but all the solutions themselves! But this is quite difficult for my problem!. It seems to me that the set of all such matrices $A$ forms a group. Is this group somehow related to the Jacobian?
I have not found anything in the literature addressing these types of questions. Any help/guidance is appreciated.