for my work I made an assessment in which users have to stack blocks in a certain configuration in as few steps as possible. If the user does this in the minimally required number of steps, they get the perfect score of 0. With every extra step they need, their score increases by 1.
To measure the skill of the participants, we use an IRT (item response theory) model. Since the test isn't dichotomous, scores are discrete, scores on every item can range from 0 to $\infty$, and scores seem roughly normally distributed, we adapted the IRT model to use a poisson distribution.
That is, we use the normal MaxLogLikelihood estimation, but instead of using the 2-parameter IRT fomula(1) $$ P(X|a,b,\theta) = {e^{a(\theta - b)} \over 1+ e^{a(\theta - b)}} $$
we calculate P according to the regular poisson formula (2) $$ P(X=n) = {\lambda^n e^{-\lambda} \over !n} $$
where $\lambda$ is... well, that's the question. Based on a textbook I have been using (3) $$ \lambda = e^{(b - a\theta)} $$
My questions are about the relationship between $b$, $\lambda$ and $\theta$:
- does the formula at (3) make sense, or should it be something else?
- in the 2plm IRT model (1) the roles of the parameters are very clear to me: $\theta$ is assumed/defined to be N(0,1) so ranges roughly from -3 to 3. $b$ is defined such that if $b = \theta$, the chance of getting a correct answer is exactly 50%. And $a$ defines the slope. Makes perfect sense. In the poisson model however, I lack these definitions. If $b$ still represents the difficulty of an item, and suppose $b = \theta$, what $\lambda$ should I expect? What does $a$ represent? I'm finding it hard to wrap my head around the conceptual meaning.
In short: keeping $\theta$ defined as N(0,1), what would be the best/most sensible way to define the other parameters of a poisson-based IRT model?
p.s. first post here, feel free to give other feedback.
If you use the more commonly used IRT representation:
$$ \lambda=e^{ a(\theta-b)} $$
and you fill in $b=\theta$, you obtain:
$\lambda=exp( 0 )= 1$. Thus, an item with a $b$ parameter equal to the average $\theta$ leads to an item that a person gets right on on average 1 try more than is absolutely necessary.
For larger $b$s, the exponent gets smaller (e.g. $exp(-100)$) so the number of tries decreases to nearly 0 (practically perfect scores for everyone.)
For smaller $b$s, the exponent gets bigger (e.g. $exp(100)$) so the number of tries increases, i.e. the item becomes very difficult.
In sum, in this IRT representation, the $b$s represent the easiness of the item.
The $a$s represent the correlation between the expected number of tries more than necessary, and the latent ability theta. For large $a$, there is a very strong relation between how able a person is and the number of tries. For an item with a very small $a$, there is no relationship between the number of tries for that item and the ability of a person.
Note though that $\theta$ represents inability, since a large value for $\lambda$ represents a large number of tries.
If you would parameterize the model as $$ \lambda=e^{a(b-\theta)} $$ you would have a $\theta$ that would represent ability instead of inability, and $b$-parameters that represent difficulty instead of easiness.
Also note that you probably don't want to fix the distribution of $\theta$ to $N(0,1)$. That would entail that most of your participants have perfect scores. Best to have the data tell you something about that.