What's the correct interpretation for f1 score when precision is Nan and recall is not Nan?

Question

Given the following formula:

Precision = TP / (TP + FP)
Recall = TPR (True Positive Rate)
 F1 = 2((PRE * REC)/(PRE + REC))

What is the correct interpretation for f1-score when precision is Nan and recall isn't?

Answer 1

You want to interpret $2TP/(2TP+FP+FN)$ if $TP/(TP+FP)$ is NaN but $TP/(TP+FN)$ isn't. Programming languages come in two kinds for this:

Case I: NaN results from one of the variables being NaN. By process of elimination, it's FP. Therefore, $2TP/(2TP+FP+FN)$ will also be NaN.

Case II: NaN results from a division by $0$. Then $TP=FP=0\ne FN$, so $2TP/(2TP+FP+FN)=0$.

According to these answers, C++ is case II, obtaining NaN from $0/0$.

Answer 2

Presumably the precision $TP/(TP+FP)$ is coming as NaN because $TP=0$ and $FP=0$:

• If there are no false negatives either, i.e. $FN = 0$ too and the sensitivity is also NaN, then the $F_1$ score really is undefined: there were no cases where your model had any opportunity to identify any true positives whether it was good or not, though at least it had no false negatives. Sine the $F_1$ score ignores true negatives and treats them as irrelevant (so it is not really a complete accuracy measure), you have no useful data for calculation of $F_1$.

• If there are any false negatives, i.e. $FN \gt 0$, then you can take the $F_1$ score to be zero: your model failed to identify any true positives when it could have done. The way to see this is to note that you can write $F_1 = \frac{2\, TP}{(TP+FP) +(TP + FN)}$.

• You can use the same argument as the second point to deal with the case where the sensitivity is NaN, i.e. $TP=0$ and $FN=0$ but the precision is $0$ because $FP>0$. Your model identified positives when there was no possibility of true positives, so you can take the $F_1$ score to be zero.