In the following function I want to find the complexities.
f(n) = nlog(64n!) + n^2√n Use Stirlings Approximation
(a) f(n)∈O(n^3 )
True. The upper bound of the function is f(n)=n^(5/2) and it is less than n3.
lim f/g = 0; It is also o(n^3).
(b) f(n)∈Θ(n^3 )
False. n^3 is bigger than n^5/2.
lim f/g = 0;
(c) f(n)∈Ω(n^2 log(n))
Actually False but it is true for some values of n.
lim f/g = ∞;
(d) f(n)∈ω(n^2 √n)
False. Because lim f/g is not infinity.
f = o(g) if lim f/g = 0,
f = O(g) if lim f/g ∈ [0,∞),
f = Θ(g) if lim f/g ∈ (0,∞),
f = Ω(g) if lim f/g ∈ (0,∞].
f = w(g) if lim f/g = ∞.
a,b and d is clear to me but I don't understand the option c
I draw the graph of the function as follows

For some values of n, n^2logn can be larger than n^5/2 How can the average complexity be determined for this function?
$f(n)\in\Omega\left(g(n)\right)$ if there exists some $n'$ and some $\epsilon>0$ such that $f(n)>\epsilon\cdot g(n)$ for all $n> n'$.
Basically, $f(n)$ must be within a constant multiplicative factor of $g(n)$ for all sufficiently large $n$, i.e. "from some point onwards". What happens for "small" $n$ is irrelevant; only what happens as $n$ grows to infinity matters - that's why it's called asymptotic notation.