"F1 score better" equivalent to "accuracy better"? Or not?

28 Views Asked by At

I have been asked to proof:

Suppose we have two models, $M1$ and $M2$, that solve the same binary classification problem for which we compute the $F_1$ values and accuracies. I have been asked to prove:

$$F_1^{M1} \leq F_1^{M2} \iff \text{accuracy}^{M1} \leq \text{accuracy}^{M2}$$

However, I think I found a counterexample... Did I do something wrong?

Counterexample

Suppose we have 5 real positive and 4 real negative cases, which we try to label with Model 1 and Model 2.

We construct Model 1 such that it behaves as $TP_1 = 3$, $TN_1 = 2$, $FP_1 = 2$, $FN_1 = 2$.

We construct Model 2 such that it behaves as $TP_2 = 4$, $TN_2 = 0$, $FP_2 = 4$, $FN_2 = 1$.

(TP is true positive, TN is true negative, FP is false positive, FN is false negative).

Consequently, we have, $F_1^{M1} = 0.6 < 0.615 = F_1^{M2}$

and

$\text{accuracy}^{M1} = 0.556 > 0.444 = \text{accuracy}^{M2}$

which contradicts the statement. Huh?