I have been doing tests doing side channel attacks test for my thesis and my test results are unclear. I have measured two distributions, one for a valid and one for a non-valid input to a system. The distributions are different (obviously) but not very much.
What I am wondering:
How can one calculate the probability of a third distribution being either of type one or two? I neither know what principal I would have to look for nor how to call this. An attacker would use this to determine if their input was of type "valid" or "non-valid". I want to know how
The only glimpse I have is what I barely understand about cross-entropy as a loss function in machine learning which uses the Kullback-Leibler divergence. As I understand it can be used to compare two distributions and will yield a similarity of some sort, so I should be able to compare the similarity of distribution number three to each previously obtained distribution and at least determine to which it is more similar. But that wouldn't tell me how likely that is correct assumption.
I have also never had to apply it by hand (so to say) because I have been using Keras doing machine learning.
I would love for an explanation of even just a pointer as to what that is called I am trying to find out, so I can research it. Please also suggest improvements to my title as people coming in might have expected what I am asking
Assuming your distributions are discrete, you can try different approaches.
For example, Kolmogorov Smirnov test
Please see Goodness of fit in general and chi-square goodness of fit test in particular.
A nice discussion is here