This is a problem from a university exam:
True or false: $n^{1.1} \in O(n(\log n)^2)$.
The solution says False, but I'm unable to prove it. I tried using the limit test for Big-O:
$\lim_{n \to \infty} \frac{T(n)}{f(n)} = \lim_{n \to \infty} \frac{n^{1.1}}{n(\log n)^2} = \lim_{n \to \infty} \frac{n^{0.1}}{(\log n)^2}$
EDIT: Wolfram Alpha says that this limit is $\infty$, which is not a constant as the limit test requires. But I still don't know how to compute this limit by hand.
More basic looking, by L'Hopital $$ \lim_{x \rightarrow \infty} \frac{x}{\log x} = \infty $$
Next $$ \lim_{x \rightarrow \infty} \frac{x}{(\log x)^2 } =\lim_{x \rightarrow \infty} \frac{x}{2 \log x } = \infty $$
The general induction step is $$ \lim_{x \rightarrow \infty} \frac{x}{(\log x)^k } =\lim_{x \rightarrow \infty} \frac{x}{k (\log x)^{k-1} } = \infty $$
For non-integer exponents comparison may be used. Anyway, your $$ \left( \frac{n^{0.1}}{(\log n)^2} \right)^{10} = \frac{n}{(\log n)^{20}} $$ and both go to infinity.