F1-score giving good value on imbalanced dataset.

Question

If I have an imbalanced dataset that consists of 90% positive points and 10% negative points. Now I created a "dumb" model which always predicts every point as a positive point. The confusion matrix of this problem will be -

enter image description here

Now the Precision of the above confusion matrix will be -

Precision = No. of true positive out of the number of points predicted positive by the model.

So, Precision = TP / (TP + FP) = 90 / (90 + 10) = 0.9

And Recall will be -

Recall = No. of true positive out of the number of points that are actually positive.

So, Recall = TP / (TP + FN) = 90 / (90 + 0) = 1.0

As we can see both precision and recall are high and F1score of this model is,

F1-Score = PR / P + R * 2 = 0.94

This is a high score, so according to this, the model is very good. How can the f1 score give good results on an imbalanced dataset?

Answer 1

Your $F_1$ score is high because both precision and recall (for the positive class) are high. Note that $F_1$ is specific to one (positive) class (in a binary classification problem). In a multiclass problem you would need to calculate precision and recall for each class separately and then aggregate them.

Try, in your example, to calculate the recall of the negative class (this is referred to as true negative rate/specificity):

$$ TNR = \frac{TN}{N} = 0 $$ i.e. your model is unable to recognize (recall) any of the negative samples. Imagine if you "swapped" the labels (positive class with negative class) and classified all of your samples as negative - your model's accuracy would still be 90% but your $F1$ score would be zero.