$$x^TAx+x^Tb+c=(x-h)^TA(x-h)+k$$ where $$h=-(A+A^T)^{-1}b$$ $$k=c-h^TAh$$
Determining $h$ and $k$ above is called "completing the square" and requires matrix addition, inversion, transposition, multiplication etc.
I am implementing algorithms (such as the above and others) in a generic way such that they can apply to any entity on which the requisite operations (addition, inversion, etc.) are defined.
I would call this library "matrix algebra" but I wondered whether there is a more abstract/generic name for "something that can be added, inverted, transposed, multiplied, etc.")?
Yes this would be an example of geometric algebra. A matrix is just a representation of general algebraic /geometric objects like vectors, multivectors, dual multivectors etc..