I have a linear equation as follows:
$B_0*x_0 + B_1*x_1 + ... + B_8*x_8 = result$
And i have about 200 different situations that are categorized into two different groups, depending on whether result is greater than or less than 1. I.e.
$B_0*x_0 + ... + B_8*x_8 = 1.34$ (for situation 1) $$..$$ $$..$$ $$..$$ $B_0*x_0 + ... + B_8*x_8 = 0.25$ (for situation 200)
If I know the expected result for each situation, how could I optimize (through some sort of regression) the weights Bi to achieve close to 100% accuracy? I also know the $x_i$'s for all of the situations. Ideally, I'd like to solve for a set of $B_i$'s that would work for any set of of $x_i$'s.
Any suggestions would be sincerely appreciated.
If we label the RHS (results) $r_i$ and the corresponding column vector $r\in\mathbb R^{200}$ and the matrix $X$ where $X_{ij}$ is $x_j$ of the $i$-th sample ($X\in\mathbb R^{200\times 8}$), we want to find a vector $b$ such that $$Xb=r$$ Since this is overdetermined, we must minimize the error. The usual way to do this is to solve $$b = \mathop{\rm arg\;min}_{y\in\mathbb R^8} \|Xy-r\|_2$$ The solution to this can be found using a lot of iterative solvers (wich is numerically stable), such as LSMR, PCG or GMRES. Analytically, the solution is given by $$X^TX b = X^T r \Leftrightarrow b = (X^TX)^{-1}X^T r = X^\dagger r$$ With the pseudoinverse $X^\dagger$. This however is numerically unstable so it shouldn't be used to actually compute $b$.