What would the big $O$ (worst-case runtime complexity; I think it's big $O$?) be for an algorithm that takes this long? I generalized the run time with the summation and put it in wolfram alpha.
$$\sum_{i = 0} ^{\sqrt n} i \sqrt n = \frac{1}{2} (\sqrt n + 1) n$$
I assume the actual run time would be the value on the right, so the big $O$ would be $n^{3/2}$? Please let me know if this is unclear. Thanks.
Yes, you're correct. If the runtime is as you claim 12(n√+1)n, then it is O(n3/2) (and this is in fact easy to verify: by showing that 12(n√+1)n⋅n−3/2 tends to a finite (nonzero) value, you will show that in fact 12(n√+1)n is assymptotic to n3/2, or, by definition, is O(n3/2)).