I am trying to model a POMDP based on user feedback dialogue and an observation set based on eye gaze. The goal is to specify which item the robot should pick up according which item the human's eye gaze falls upon.
States: Agent is {clueless, possible, certain} about which item human wants.
Actions: Agent can {ask human, hover over item, pickup item, put down item, hand over item}
Observation Set: Eye gaze can fall upon areas A B C (each corresponds to an item)
How can I: 1) Incorporate the observation set into my model and 2) Update the transition probabilities given the human's answer of "yes" or "no" when robot asks "is this the correct item"?