I was looking at this paper (https://arxiv.org/pdf/2005.14165.pdf) and came across this graph:
I am trying to understand the following two things about this graph:
What is PetaFLOP/s-days? I read that a PetaFLOP is 1,000,000,000,000,000 calculations (e.g. addition, subtraction). I am guessing that 10^2 would imply 100 * 1,000,000,000,000,000 calculations per day - is this correct? Is there any difference between PetaFLOP/days and PetaFLOP/s-days? (I also find it interesting they are probably referring to "computer resources" as simply "compute")
What does "C" stand for in L = 2.57 * C^-0.048? I am guessing that the "dotted line" probably refers to the "average loss" for different Neural Networks with differing amounts of Parameters - but what exactly does "C" stand for?
Finally, is there a reason that "Validation Loss" is not expressed as a percentage? For instance, what is a Validation Loss of 3? Is a Validation Loss of 3 the same as a Loss of 30%? Or does Validation Loss simply refer to the value of the Loss Function obtained during the Validation stage of Cross Validation?
Thank you!