Our numerical analysis textbook has this problem:
How many bytes does it take to store $100$ vectors of length $10^5$? How many flops does it take to form a linear combination of them (with $100$ nonzero coefficients)? About how long would this take on a computer capable of carrying out $1 Gflop/s$?
Okay, so first off, my confusion is regarding whether I should be interpreting the "Giga" prefix to mean $10^9$ or its rough binary approximation of $2^{30}$. That makes a difference, right? So, here's my attempt:
Part 1
Assume we're working with a $32$-bit machine and that each element of a vector requires $4$ bytes. A single vector has $10^5$ elements, meaning it requires $4 \times 10^5$ bytes. We have $10^2 = 100$ of these vectors, giving us a total of $4 \times 10^7$ bytes ($40,000,000$).
(Would this be $2^{25} =33,554,432$ in binary?)
Part 2
Okay, this one was the toughest in my opinion. Please let me know if my work is correct.
Suppose we have this linear combination:
$a_1\vec{v_1} + a_2\vec{v_2} + ... + a_{100}\vec{v_{100}}$
Now, let's focus on just the first term:
$a_1\vec{v_1}$ requires that you distribute the constant to $10^5$ terms. This is $10^5$ flops.
Now, extending that, we have $100$ of these scalar-vector terms. So that's $10^2 * 10^5 = 10^7$ flops.
Moreover, once we have done that, we have to add each resulting vector's terms in a component-wise fashion.
This requires that we add the first component of $\vec{v_1}$ with the first component of $\vec{v_2}$ with the first component of ... $\vec{v_{100}}$.
In other words, we have to add $100$ components, but each vector has $10^5$ components "stacked" vertically. So that's another $10^2 \times 10^5 = 10^7$ flops.
Giving me an answer of $10^{14}$. I am not sure if this is correct. Moreover, I'm not sure if $2^{46}$ is the correct binary approximation.
Part 3
This one's straightforward, but it assumes my answers to the first two parts are correct.
Also, I am again using an approximation of $2^{30}$ for the giga term.
I got $\frac{2^{46}}{2^{30}} = 2^{16} = 65,536$ seconds.
Note: If I leave things in their decimal form, I get an answer of $10^5 = 100,000$ seconds. So I guess that's sort of close?
It seems to me the way you are proposing this solution struggles in two ways. For one thing, some of your calculations are a bit off, and for another, binary to decimal conversions are a little tricky.
Maybe for this sort of problem it would be easiest to simply put everything in the same base, and convert later. Let's assume decimal because the problem is entirely worded that way, except for mention of bytes. You can convert to binary if that's explicitly required afterwards. See this (very old) link for a good discussion how.
I would recommend you solve this problem entirely mathematically, and I'll give you the mathematics answer since we are on this part of SE.
Part 1
This part looks fine to me as it is. You've even been thorough to move from element to vector to number of vectors.
Part 2
This also almost looks good to me.
Good point, though technically you are only adding 99 times, about the same. But then you say
It's not, because you aren't adding separately for all multiplications. In fact what you are doing is multiplying them all, and then adding. This is like $ O(n^2) + O(n^2) $ and not $ O(n^2) * O(n^2) $. So you get $ 10^7 + 10^7 = 2 * 10^7 $, where one is from adding and one from multiplying.
Part 3
This answer doesn't make sense because of the incorrect estimate from Part 2. It may help to use units when you have to measure times as well as data, but your method is correct. You should get an answer of $ 10^7 / 10^9 $ seconds, assuming a Gigabyte is a billion bytes, which it often is less than in storage practice.