Precision and Recall of spelling corrector

Question

I am working on an automated spelling corrector. Now I have to evaluate it. Therefore I need Precision and Recall and it is absolutely clear how to calculate these. My problem is that I am not sure what True Positive, False Negative, etc. is.

I think there are two options e.g. for TP (for the other 3 analogous):

a) The number of wrong written words which were recognized as wrong

b) The number of wrong written words which were corrected into the intended word

I think a) wouldn't say anything about the quality of the correction, but it fits most my intuation of TP.

Best

C.Klaus

Answer 1

A true positive is a wrong word which is properly recognized as wrong, and a true negative is a correct which is properly recognized as correct (and therefore not corrected by your algorithm). The other two are a bit more confusing, but from Wikipedia,

A false positive error, or in short a false positive, commonly called a "false alarm", is a result that indicates a given condition exists, when it does not. For example, in the case of "The Boy Who Cried Wolf", the condition tested for was "is there a wolf near the herd?"; the shepherd at first wrongly indicated there was one, by calling "Wolf, wolf!"

and

A false negative error, or in short a false negative, is a test result that indicates that a condition does not hold, while in fact it does. I.e., erroneously no effect has been inferred. An example is a truly guilty prisoner who is acquitted of a crime. The condition "the prisoner is guilty" holds (the prisoner is guilty). But the test (a trial in a court of law) failed to realize this, and wrongly decided the prisoner was not guilty, falsely concluding a negative about the condition.

Thus, for example, FN represents all the wrong words which your spell checker missed.