Gamified Psychophysics
Since 2022 I am designing, programming, and testing psychophysical experiments that incorporate gaming elements, to study Cognition, Motion Processing, Decision-Making, Integration of Social Information.
Four major projects sparked from this effort:
An investigation on the validity of gamified psychophysics for the study of motion processing;
A dyadic, joystick-based game for to study the integration of social information in decision-making;
A multiple-choice user interface optimised to achieve cognitive assessment, enrichment, and psychophysics of monkeys operating touchscreen from their own enclosure.
A foraging-like game to study high-level cognitive flexibility in monkeys and humans (adults and children with and without ADHD)
An investigation on the validity of gamified psychophysics for the study of motion processing: can game elements be used in psychophysics?
The short answer is Yes!
Across three experimental conditions with gradually different levels of gamifications (Standard = no game elements, Gamified = simple, 2D gamified elements, Unreal Engine = complex, 3D gamified elements), where human participants played the exact same task, motion processing, subject satisfaction, and dropout rates were assessed.
Participants were asked to move the joystick left or right depending on cloud of dots at the top of the screen. The triangle represented their avatar which would jump left and right, onto dedicated lanes, depending on the joystick position. Moving the joystick forward or backward would make the avatar accelerate or decelerate. At random points in time, obstacles would appear on the lane, too fast for the player to avoid, which upon collision would deplete the player energy bar, eventually causing death and the restart of the level. Obstacles could be avoided only by mimicking the motion of the cloud of dots. Every successful jump would provide points which filled the score bar on the left and eventually lead to the next, more difficult level, with different color background and noisier cloud of dots!
Participants were told that their mind was imprisoned by an evil force into an infinite obstacle course. An AI working in the prison took pity on them and decided to help, but help must be hidden! Thus the AI would comunicate to the player where to jump, left or tight, by manipulating a cloud of dots. Unfortunately the more participants approached the exit of the prison, the more obstacles there were and the more perturbed the AI dots cloud would become.
Across the 30 participants (10 per condition), individuals who performed the Unreal Engine condition improved significantly more in their motion perception (compared to before playing the game) compared to individuals who played the standard psychophysics task and without significant difference from those who played the gamified task (left figure above).
From a sensory perspective, the three conditions produced no significant difference in reaction times, suggesting that the psychophysics evaluation of the cognitive processes behind the experiment were not dissimilar!
…gamification also increased the rate of participation.
… and the participants’ satisfaction …
A dyadic, joystick-based game for to study the integration of social information in decision-making: is social information integrated in sensory processing?
The short answer is Maybe!
The data analysis for this study is still ongoing and the paper is in preparation. Contact me if you are interested in the answer!
Participants used a joystick to move an arc around a cloud of dots. The more they pushed, the smaller the arc became. At random points, circled appeared around the screen (called targets). Circles needed to be collected, by placing the arc right under the targets.
The global motion of the cloud hints at where the targets would appear.
The same task could be played by two people simultaneously, each seeing their own arc and the other player’s arc. I cannot disclose here more about the results of this experiment.
A multiple-choice user interface optimised to achieve cognitive assessment, enrichment, and psychophysics of monkeys operating touchscreen from their own enclosure: can monkeys select what game they want to play?
Of course they can! So the short answer is YES!
This study, published in 2023 in the peer-reviewed journal Animals, gave animals operating touchscreen devices from their own enclosure, the opportunity to play different game-like activities, each accessible via one fixed button at the bottom of the screen.
A sort of “dock bar” for monkeys, I called it Multiple-Choice Interface.
Animals could choose one of three activities: touching a red ball to receive sugary juice (static reach); touching a moving red ball to receive sugary juice (dynamic reach); watching pictures of other monkeys doing monkey things!
Finally, with the Multiple-Choice Interface not only animal can choose which cognitive enrichment activity to play and when to play it, but the interface also allows psychophysics evaluation of the animal performance!
For example it can be measure what is the degree of improvement across red ball size and speed. Also, it is possibile to find best combination of red ball size and speed that elicits the highest hit rate.
Monkeys understood very well the concept of “Task Bar” as most of them kept choosing their favourite task over and over despite the button changed in position every hour!
While most monkeys liked the static reach more than the dynamic, few had the opposite preference. Also, all monkeys preferred the tasks that delivered sugary juice compared to just watching pictures!
A foraging-like game to study high-level cognitive flexibility in monkeys and humans (adults and children with and without ADHD): how flexible are humans and monkeys?
The short answer is that Humans and Monkeys are flexible, but in different ways!
This study, published in 2022 in the peer-reviewed journal Frontiers in Psychology, gave animals operating touchscreen devices from their own enclosure, the opportunity to play engage with a complex cognitive assessment, optimised to provide also high levels of cognitive enrichment!
Animals liked to play this game very much!
With thousands of trials per session, dozen per minute and very few breaks during each session, their engagement varied across animals but stayed consistent within animals. Each monkey had its own play style!
As expected, humans are more flexible than monkeys and learn rules on a more abstract level.
Nonetheless, rule learning and switching in both humans and monkeys does not depend on the specific visual domain of the rule!