Algorithm analysis - when to throw away terms?

59 Views Asked by At

In this algorithmic analysis of least squares regression, we throw away big-$O$ terms that will be dominated by the biggest, and keep only the dominant term.

On the other hand, in this algorithmic analysis of matrix multiplication followed by truncation, the matrix multiplication dominates the truncation, but we keep the lesser big-O term, and the final result is the product of both.

Why is this? When should we throw away non-dominating terms, and when should we keep them?

2

There are 2 best solutions below

2
On

In the latter, $T$ requires $O(m)$ work for each of the $O(mn)$ entries of $A\bf y$. That’s why they are multiplied.

0
On

We always could throw away non-dominating terms.

For the second Q, it's the same.

Given that the truncation operator :$ℝ^→ℝ^$ by $()=max\{min\{,\},0\}$, it's reasonable to deduce that it takes $O(m)$.

Matrix multiplication takes $O(m n)$, and then we get a new vector Ay, applying T on it takes another O(m).

Hence, it takes O(mn+m) = O(mn) in total, throwing away the non-dominating term.

(Exception is that it's done in designated hardware, where it could be parallelized, as low as O(1) for truncation operator.)