June 18, 2004
Insight into the Brain's Machinery of Value
New research with monkeys has yielded insight into the neural
machinery that animals, and possibly humans, use to mentally represent
the value of one action over another. Scientists at the Howard Hughes
Medical Institute (HHMI) have determined the strategy by which the
brain calculates the value of potential behavioral choices as those
values change over time with new experience.
Their studies, said the cognitive neuroscientists who conducted the
research, help in understanding the mysterious process by which animals
and humans process sensory input to decide on actions that yield the
greatest advantage.
“These monkeys had to make accurate probability estimates, and we found that the bits of math going on in their brains are quite impressive.”
William T. Newsome
The researchers, led by HHMI investigator William Newsome at the
Stanford University School of Medicine, published their findings in the
June 18, 2004, issue of the journal Science.
“Researchers who study behavior regard decision-making as a
critical link between the classic fields of the study of perception and
the study of motor output,” said Newsome. “Sensory
information comes into the brain, and somewhere that information is
evaluated and decisions get made about what's out there. For example:
Is it a predator? Is it a prey? Is it food or another
object?”
In their studies, Newsome and co-authors Leo Sugrue and Greg Corrado
set out to explore the machinery of value by asking rhesus monkeys to
perform an eye-tracking task to get a fruit juice reward. Newsome said
the researchers designed the task to probe deeply into the complex
mechanism the brain uses to encode the value of choices.
The monkeys were given a choice of looking at either a red or green
target to get a sip of juice. For each target, the experimenters
changed the odds of receiving the reward at unannounced times. The
monkey could receive the most sips of juice by adjusting its choices
over time to match the changing frequency of rewards. Importantly, the
most effective strategy required the monkeys to periodically `check
out' the target with the lower odds to detect the unannounced changes
in the reward rates. By analyzing the statistics of the choices over
time, the researchers gained insight into how the monkeys tracked the
targets' changing values, said Newsome.
“Psychologists and economists have known for a long time that
decision-making is far more complex than just receiving sensory
information from the environment and basing an action on it,”
said Newsome. “You build up a history over time of what the
likely outcomes of different actions are and build up these internal
representations of value. These internal estimates of value — or the
likelihood of acquiring your reward — influence decision-making as
much as sensory evidence, and sometimes more so.”
For example, said Newsome, a fly fisherman will decide where to fish
along his favorite stream based on a decision-making process that
includes a complex judgment of immediate sensory input — like the
weather and the water conditions — as well as a multitude of past
experiences in fishing the stream.
After analyzing many hundreds of trials with the monkeys, Newsome
and his colleagues found that the animals were, indeed, making
sophisticated, flexible judgments about where to look to get the
reward. They adapted to even subtle changes in the statistical
likelihood that looking at a particular target would yield a sip of
juice.
Not only did the monkeys rapidly adjust the frequency with which
they looked at one target or the other as their values changed, but
they “hedged their bets” by glancing at the less rewarding
target in case it was becoming more rewarding, said Newsome.
“These monkeys had to make accurate probability estimates, and
we found that the bits of math going on in their brains are quite
impressive,” said Newsome. “It's not math the way humans
would write equations. This is math that is built into the brain
through evolutionary history, presumably from calculating the odds of
finding food or water or a mate at different places in the
wild.”
One surprise, said Newsome, was that the monkeys relied on
information that was more `local' than studies by other researchers had
suggested. Rather than adding a new experience to the entire history of
experience with the task, the monkeys tended to give more value to the
last seven or eight trials, with the influence of earlier trials
dropping off rapidly beyond that.
Such a local valuation reveals an important tradeoff, said Newsome.
If the monkeys used a longer history, they might be more accurate, but
they would respond sluggishly to changes. On the other hand, if they
used only very recent information, they would respond more quickly to
change, but would make more mistakes when the choice system was
stable.
The researchers found the same localization phenomenon when they
developed and ran a mathematical simulation of the monkeys' choice
behavior.
Newsome and his colleagues also used recording electrodes during the
behavioral trials to explore whether such decision-making was taking
place in the area of the brain called the parietal cortex, which
contributes to decisions about where to move the eyes. Their
electrophysiological studies revealed patterns of neuronal activity
corresponding to the value estimates the monkeys indicated by their
target choices. This finding provided an initial clue about how and
where value is represented in the brain, and how value signals might
influence decision-making.
However, said Newsome, analysis of parietal cortex activity in the
brief interval between behavioral trials suggested that the brain did
not store value information in that area. Rather, the value appeared to
be “downloaded” at the start of each trial from some other
part of the brain. Newsome and his colleagues plan further studies,
including magnetic resonance imaging studies, to pinpoint these
regions.
Such a combination of behavioral and neurophysiological studies and
mathematical modeling offers an important new route to insight into the
neural machinery of valuation and decision-making, concluded Newsome.
Also, he emphasized, such studies depend on the decades of studies of
both sensory and motor systems in the brain.
“This whole area of decision-making studies is exploding right
now, largely because for the first time in history we know enough about
the sensory and the motor ends of the system to put ourselves in
position to ask more reasonable questions about the key intermediate
stage of decision-making,” he said.
|
