Why is the following true?
$3n^2-100n+6$ is big $O$ of $n^2$
This has been demonstrated to be true when $c$ is $4$ and $n$ is $10$.
$3*100-1000+6 = -694 = 694$ is the absolute value is a big $O$ of $4*100 = 400$
It looks like $n^2$ is a lot less. What am I doing wrong?
It seems like you are confused as to the definition of "big $O$". (either that or I'm confused)
$f(n)$ is big $O$ of $g(n)$ means $f(n)/g(n)$ is bounded. Since in your case $f(n)/g(n)=\frac{3n^2-100n+6}{n^2}$ converges to $3$, it is bounded.
I don't see where "$c$" comes into it.