HHMI researchers have identified a brain pathway that plays an important role in making survival decisions.

Mouse with cheeseResearchers have identified a brain pathway that plays an important role in making survival decisions—such as a mouse's decision to flee rather than try to mate when confronted by a cat.

The study offers new insight into the neural circuits that govern decision-making when critical behavioral choices must be made, said senior author David J. Anderson, a Howard Hughes Medical Institute investigator at the California Institute of Technology.

Anderson and his graduate student Gloria Choi collaborated on the study with researchers from the University of Southern California and Regeneron Pharmaceuticals in Tarrytown, N.Y. Their paper was published in the May 19, 2005, issue of the journal Neuron.

The researchers sought to study the genetic specification and function of neural pathways that connect the amygdala and the hypothalamus. The amygdala governs emotional processing of environmental stimuli. The hypothalamus produces hormones that regulate behaviors such as mood and sex drive. Classical neuroanatomical studies had shown that there are parallel pathways from the amygdala to the hypothalamus that mediate innate behaviors that are usually mutually exclusive.

“These two parallel pathways connecting the medial amygdala and different parts of the hypothalamus were known to control reproductive or defensive behaviors,” said Anderson. “Reproductive behaviors include not only mating behavior but also maternal behaviors, for example. And defensive behaviors include not only running away, freezing, or predator response, but also potentially aggressive responses as well. The two distinct neural pathways between these two brain structures were known to be activated by odorant chemical signals called pheromones that are detected by a specialized olfactory structure called the vomeronasal organ, located in the mouse's nasal passages, said Anderson.

“Since these pathways are anatomically segregated, it suggested to us that there must be genes that lay these circuits down during development, as part of the genetic basis for specifying how these innate behaviors are programmed by the genome,” he said. “So, we were interested to see if we could identify genes that, at the very least, might mark these pathways, and at best, might be involved in specifying their actual wiring or assembly during development.” 

The researchers also sought to determine how the animal integrates behaviors governed by the two pathways. “If you have segregated parallel pathways that mediate either reproductive or defensive behaviors, how does the animal decide what to do when it's faced with conflicting cues for both types of behaviors at the same time?,” asked Anderson. “What if it encounters a predator threat at the time that it's supposed to be protecting its young or about to engage in mating?” 

The researchers microdissected brain tissue from the two pathways under study and subjected the neurons to genetic and axonal tracing analyses. These studies revealed that the pathway involved in reproductive behavior was distinguished by the expression of a particular gene called Lhx6 . This gene is a transcription factor that controls the activity of other genes.

They next conducted olfactory stimulation experiments to confirm that this pathway was, indeed, activated by the appropriate odorant cue. They exposed male mice to either female urine—which contains pheromone signals that triggers behaviors relevant to mating—or to a predator odor derived from a domestic cat, which can trigger defensive behaviors such as freezing. They found that neuronal activity in the Lhx6 -marked neurons was triggered by the former, but not the latter odor.

The researchers also found that two other related genes, Lhx9 and Lhx5, marked pathways that are potentially associated with defensive or aggressive behavior. Anderson said, however, that their axonal tracing studies and the activation of neurons by the cat-collar odor did not give a straightforward association between this circuit and predator-defense behaviors.

“Since those neurons are not activated by the cat-collar odor, we can't say that they are involved in predator defenses,” said Anderson. “We have indirect evidence that these neurons might be involved in aggressive behavior, but we can't be certain. That is something interesting to explore for the future.”

The researchers' analyses of the function of the two circuits also yielded insights into how they might interact to influence the hypothalamus in controlling reproductive behavior in the presence of cues that require a defensive response. They found that the parallel pathways from the amygdala activated by cat-collar odor or female urine converge on a part of the hypothalamus that governs reproductive behavior. The pathway activated by cat-collar odor has the opposite functional effect as that activated by female urine, suggesting that it antagonizes the latter. In this way, the two projections may combine to create a “gate control” mechanism that can inhibit reproductive behavior in the presence of a threatening stimulus such as a predator.

Anderson emphasized that his group's findings suggest that there is still a lot of unexplained complexity in this mechanism. Thus, they are planning additional behavioral experiments, such as testing the animals' responses to simultaneous reproductive and threat stimuli. The answers will help the researchers explore in more detail how the mechanism works to integrate the two conflicting signals in the mouse's brain.

This “love-or-war” olfactory-triggered circuit might also exist in humans and other primates, said Anderson. “Humans don't have a vomeronasal organ, but there is increasing evidence that humans and non-human primates sense pheromones. It is possible that their medial amygdala might be activated by the types of olfactory cues that in the mouse would activate the vomeronasal organ. So, the lack of a vomeronasal pathway in humans and non-human primates doesn't rule out the existence in these species of the kind of gate that we propose. But it would require that the sorts of stimuli that we are studying go through the medial amygdala, and there is very little evidence for that at this point.”

Scientist Profiles

For More Information

Jim Keeley 301.215.8858