Capturing Behavior

To mathematician and computer scientist Kristin Branson, large sets of data, whatever their source, are vast mines to be explored for the great value they might hold. Now a Janelia Farm fellow, Branson has set out to teach computers how to discern the digital hallmarks of animal behavior within massive troves of zeros and ones that make up video recording of fruit flies doing what they do. “I have been watching fruit flies for the past five years,” she says with a wry smile. The overall goal is to identify changes or characteristics in behaviors, such as grooming, chasing, courtship, light-driven movements (phototaxis), and aggression, which are linked to specific brain traits. These traits, in turn, are associated with the thousands of fly genotypes that Janelia Farm is famous for having on hand and that are well represented among the flies Branson’s group has observed.

Branson works with biologists who watch the captured video footage and search for behaviors they can confidently identify. Among the 60 or so descriptive parameters that they’ve coded as “behavior identifiers” are such options as whether one fly is touching another or the speed at which a fly is walking in its confined laboratory environment. Each of these parameters corresponds to certain mathematically describable pixel patterns. Once in hand, Branson can use a behavior identifier to train a computer to search the growing data set for specific behaviors—the way a computer might apply a keyword search term to flag all instances of the word in a large text file or eBook.

Their work places Branson and her colleagues in the loop of an emerging field called high-throughput ethomics. In this area of research, scientists rely on large-scale observations of many individual organisms to identify and then associate particular behavioral traits with specific genetic variations. Branson’s contribution to this nascent field has been to deploy large-scale observation and data recording—in her case, video imaging of fruit flies whose genotypes differ in subtle but known ways—to discern linkages between the fruit flies’ behaviors and their genotypes.

“We have been collecting data eight hours a day, three days a week, in eight setups running in parallel,” Branson says—all in a warm, climate-controlled room the size of a walk-in closet. “We are trying to encode every behavior that biologists are sure they can identify in the video frames.” Branson and her colleagues also aim to connect these behaviors to specific neural circuits in the flies’ brains. “Once you have a behavior effect connected to a neural circuit, you are off and running,” Branson says.

-- Ivan Amato
HHMI Bulletin, Fall 2012

Return to main article >