To describe an animal's behavior, researchers observe its actions and reactions, often in response to cues in the environment, over time. That's easy enough to do with one animal or even a few of them, but how does a researcher spot different behaviors among thousands of animals? Or detect very subtle differences in the behavior of a few. "If you see one fly that walks and one that doesn't, you can easily pick out that behavior," says Kristin Branson, whose work focuses on the fruit fly Drosophila melanogaster. "But if what if one fly makes 20 percent more turns while walking than another? That is harder to recognize."

Such challenges are why researchers are turning to computers to do the job.

Branson is developing machine vision and learning programs to define and quantify key behaviors in large numbers of flies and connect them to particular neuronal circuits and genetic pathways.

This field of computer vision—a marriage of computer science and applied mathematics—is perfect for someone like Branson, who started her undergraduate studies as an astrophysics major and then switched to computer science. "I always enjoyed using math to find solutions to problems," she says.

During her graduate studies in computer science at the University of California, San Diego, Branson began turning her attention to harnessing the power of computers and mathematical algorithms to solve questions in biology. While working in a laboratory focused on machine learning—the science of getting computers to recognize complex patterns in data—she collaborated with biologists in the university's animal care facility. She designed a computer vision program for them to track and monitor the positions of mice in cages.

Branson continued with the same line of research as a postdoc in the labs of Pietro Perona and Michael Dickinson at the California Institute of Technology—this time working with more sophisticated computer programs and tracking the positions of flies instead of mice. Perona, Dickinson, Branson, and their colleagues built freely available software called Ctrax (http://dickinson.caltech.edu/ctrax) that converts infrared video of up to 50 walking flies into movement data. The software then analyzes the data to provide a quantitative description of the animals' behavior. They found, for example, that female flies can be distinguished from male flies by how often they turn—a difference in behavior that researchers had not previously detected by observing flies.

Branson plans to develop programs that allow computers to pick out behaviors of increasing complexity in flies. The way the process works is to use a biologist's knowledge of what a particular behavior looks like to train the computer to recognize this behavior. "Basically you teach the computer, when wings move like this, the fly is grooming," explains Branson. "The machine learns to recognize what is grooming and not grooming."

She will also create programs to let a computer identify behaviors that biologists have not yet discovered—a process known as unsupervised learning. The data sets collected by tracking and behavior detection software contain massive amounts of information—more than could ever be visualized by a human expert. The data can then be analyzed using data mining programs to identify patterns that might define a particular behavior. "This is something that would not be possible to do by hand," says Branson.

At Janelia Farm, Branson will continue to develop animal tracking and behavior detection software that can be adapted to different experimental systems. "Researchers can choose the different components they want to use," she says. "I am working on making these programs accessible to people who are not computer vision experts."

Several groups of researchers at Janelia have created thousands of strains of flies with mutations in different genes and are already working to link these genes to different circuits in the fruit fly brain. Branson plans to join the effort by providing ways to screen for different behaviors in all these strains of flies. "If we find a set of behaviors that can distinguish among the thousands of flies with different genetic mutations, we can then link these behaviors to different neuronal and genetic pathways," she says. "Janelia is the only place where you can do experiments on this kind of scale."