Measuring statistical consistency and reliability

670 Views Asked by At

I am trying to score reliability (or variability) for a machine learning model I'm working on and I cannot quite get my head around it. As an example, I have some people who perform a task three times per week for 10 weeks. I have other people who perform the same task, say, 10 times for three weeks during the 10 week period. And maybe I have people performing the task 30 times in one week and resting over the other nine weeks.

They all average 3 tasks per week, but I need to give them different variabilities. How would I score that?

1

There are 1 best solutions below

1
On BEST ANSWER

Simply use the standard deviation:

For your data sets, this would be \begin{align} a&=(3,3,3,3,3,3,3,3,3,3) \quad&\mapsto\quad &\rm{std} (a)=0 \\ b&=(10,10,10,0,0,0,0,0,0,0) \quad&\mapsto \quad &\rm{std} (b)=4.8305 \dots \\ c&=(30,0,0,0,0,0,0,0,0,0) \quad&\mapsto \quad &\rm{std} (c)=9.4868\dots \end{align}

Note that it doesn't matter where the $10$s (or the $30$) are placed in the list.