Exploring the role of effort and impulsivity in reversal learning

‘Exploring the role of effort and impulsivity in reversal learning’

Brief introduction

One of the main characteristics of human behaviour is its flexibility. We must be able to detect regular patterns in our environment, and also be sensitive to disruptions to those regularities, so that we can adapt our behaviour accordingly. Imagine that the bus you have been catching to work every morning for the last few years has been late for the last three days. The decision that you must make is whether this represents merely a temporary disruption of otherwise regular scheduling (say, due to road-works), or whether the bus timetable has changed altogether (such that it is no longer stopping at the previously scheduled time). In other words, you must detect whether this irregularity represents a temporary change in an otherwise stable environment, or whether it represents a fundamental change in the environment altogether.

Individuals differ significantly in how fast they learn or adapt to these sorts of situations. The way in which decisions such as this are resolved by the brain has been studied with a paradigm known as ‘probabilistic reversal learning.’ Each of these terms refer to key aspects of the paradigm:

1. Individuals are required to learn about the relative value of stimuli presented before them (e.g., which bus you prefer to catch)

2. The values of these stimuli periodically reverse (i.e., the bus timetable changes).

3. The values of the stimuli vary probabilistically (i.e., even the bus you usually prefer to catch may sometimes be late, but on average is better than the other alternative).

Using this paradigm, we now have significant insights into the neural circuitry and computational mechanisms which mediate adaptive human behaviour. Flexible learning is mediated by a neural network comprising the prefrontal cortex and basal ganglia (Clark et al., 2004; Cools et al., 2002; Izquierdo et al., 2017, Peterson et al., 2009), and dopamine is a key neurotransmitter in this process of reversal learning. Importantly, a separate literature has revealed that dopamine is important, not only in learning, but is also critical for motivating individuals to exert effortful actions (Chong et al., 2015). Given the dual role of dopamine in motivating effortful actions, and in probabilistic reversal learning, this study will explore the relationship between effort exertion and learning. Specifically, we ask:

1. How probabilistic reversal learning can differ based on the amount of force that individuals must exert to register their responses.

2. How personality differences (e.g., in impulsivity) are related to learning.

You may have been a participant in this study, but, when writing your report, it is important that you write from the perspective of the researcher.

Design

This experiment involved two phases. In an initial phase, the maximum voluntary contraction (MVC) for each participant was determined by squeezing each force dynamometer as hard as possible. Participants then undertook the learning task. Participants were presented with two abstract shapes, with one of these stimuli being more valuable on average than the other. The more valuable stimulus was rewarded 70% of the time, and the less valuable stimulus was rewarded 30% of the time. A rewarded stimulus was associated with a gain of one point, and an unrewarded stimulus was associated with no gain. The primary task was to learn which of the two stimuli was more valuable on every trial, and to accrue as many points as possible. Importantly, the relative value of the stimuli periodically reversed, such that the more valuable stimulus would then be worth less, and vice versa. Participants were instructed to detect when that change occurred, and switch their preferences accordingly. Stimuli were presented randomly to the left or right of fixation, and participants registered their preferences by squeezing the corresponding (left or right) dynamometer. Participants performed two blocks: one in which only a small force needed to be applied (5% of MVC), and the other in which a harder force was required to make a choice (30% MVC). The order of blocks was counterbalanced across participants.

Built into this design was a further experimental manipulation to examine whether the time at which participants exert a low or high force influenced learning on this task. Participants were divided into three groups (‘A’, ‘B’, and ‘C’). Participants in one group (‘A’) were only required to provide a single squeeze (with a high or low force) to register their choice and simultaneously receive feedback. In contrast, participants in Groups B and C were required to provide two squeezes. Those in Group B exerted a high or low force to register their choice, and a low force to reveal the outcome. Those in Group C undertook the reverse manipulation, by always exerting low force to register their choice, but either a low or high force to reveal the outcome. For the purposes of this lab report, you should write-up the design that you personally experienced as a participant in this study (the experimenters should have informed you of the group you were assigned to at the end of the task).Those students who did not attend the experimental sessions should write-up the design as if they were in Group A.

In order to determine the relationship between learning and impulsivity, participants were administered the revised version of the UPPS Impulsive Behaviour Scale (Whiteside & Lynam, 2001). This version, the UPPS-P, assesses five pathways: Negative Urgency, Positive Urgency, Premeditation, Perseverance, and Sensation Seeking (Cyders et al., 2007).

The key question of this study was whether there is a difference between learning rates when low or high amounts of force were applied. For the purposes of your lab report, you will answer:

1. Was there a difference in the total number of points scored during the low and high force blocks?

2. Was there a difference in the total accuracy during the low and high force blocks?

3. Was there a correlation between the total number of points scored and scores on a questionnaire measure of impulsivity?

Starting References: You will be provided with a number of papers that will provide you with some background and a rationale for the study that will form the basis of the Biological Laboratory Report. Copies of these are available on the PSY2061 Moodle website. Please print and read the following papers and bring them with you to your Week 7 lab class.

You do not need to understand the detail of the neuroimaging analyses or computational models.

Reference

Chong et al. (2015). Dopamine enhances willingness to exert effort for reward in Parkinson’s disease. Cortex, 69, 40-46. (A study showing that dopamine administration increases the motivation to exert physical effort.)

Clark et al. (2004). The neuropsychology of ventral prefrontal cortex: Decision-Making and reversal learning. Brain and Cognition, 55(1), 42-53 (An overview of reversal learning in the context of decision making and ventral prefrontal function.)

Cools, R. et al. (2002). Defining the neural mechanisms of probabilistic reversal learning using evenrelated functional magnetic resonance imaging. Journal of Neuroscience, 22(11), 4563-4567 (One of the first human neuroimaging studies on probabilistic reversal learning.)

Cyders, M. A. et al. (2007). Integration of impulsivity and positive mood to predict risky behavior: Development and validation of a measure of positive urgency. Psychological Assessment, 19, 107–118. (Background on the UPPS-P Impulsive Behavior Scale. )

Izquierdo et al. (2017) The neural basis of reversal learning: An updated perspective. Neuroscience, 345, 12-26 (A good overview of reversal learning and is neurobiology, including dopamine.)

Peterson et al. (2009). Probabilistic reversal learning is impaired in Parkinson’s disease. Neuroscience, 163(4), 1092-1101 (Examined the role of dopamine in reversal learning.)

Whiteside, S. P., & Lynam, D. R. (2001). The Five Factor Model and impulsivity: Using a structural model of personality to understand impulsivity. Personality and Individual Differences,30, 669–689 (Background on the original UPPS Impulsive Behavior Scale.)

accuracy lab_report_template__1_ cools_2002_fmri petersen_2009_parkinsons chong_2015 cyders_2007_implusivity upps_p psy2061_laboratory_report_information_and_data_collection_guide zquierdo_2017_reversal_learning points marking_rubric_2019 clark_2004_review whiteside_2001_ffm_impulsivity