1) If $a(n)=O(n^2)$ and $b(n)=O(n^3)$. Can someone tell me how to compute the computational complexity of
$$ c(n)=\sum_{k=1}^{n}a(k)b(k) $$
What rules apply?
I think it might be $O(n^6)$, but this sounds wrong.
2) What now if we have two sums. Suppose $a(n)=O(n^2)$ and $b(n)=O(1)$ and $c(n)=O(1)$ what would the computational complexity of something like this
$$ d(n)=\sum_{k=0}^{n}\sum_{j=0}^{k}c(n-k)b(k-j)a(j) $$
be? I would really like to know how to compute the computational complexity of such types of sums. Any pointers would be appreciated.
For your first question, say $a(k)\le Mk^2$, $b(k)\le Nk^3$. Then your sum is $\le MN(1^5+2^5+\cdots+n^5)$. For large $n$, the sum $1^5+2^5+\cdots+n^5$ behaves like $\frac{1}{6}n^6$. So the $O(n^6)$ is correct.