A Swiss Army Knife for Analyzing Three-Dimensional Images

For centuries, anatomists have drawn illustrations of the body’s skeleton, nerve pathways, circulatory systems, and internal organs to help determine how the different parts work together. Today, computers, microscopes and molecular genetics allow scientists to build more sophisticated, three-dimensional representations of living organisms.

To create such models, researchers must synthesize information from thousands of detailed images that reveal different information: different magnifications of a structure or different biological characteristics, such as varying gene expression across cell types. Scientists at the Howard Hughes Medical Institute’s Janelia Farm Research Campus are unveiling a new software suite packed with useful tools for visualizing, analyzing, and measuring complex, three-dimensional biological and biomedical images. The software package, called V3D, is being distributed free of charge to researchers worldwide and it promises to greatly speed up scientists’ ability to assemble and manipulate extremely detailed images, such as those of a fly’s brain.

V3D is used to visualize and reconstruct a 3D digital neuronal model of a fruit fly brain. The 3D volumetric image of a fruit fly brain is shown in magenta. The colored surface objects of irregular shapes are digital models of various brain compartments, whereas the colored tree-like surface objects are 3D reconstructed neurons. The green is a fluorescently labeled neuronal pattern.

Video: Hanchuan Peng

Janelia Farm researchers and their colleagues used V3D to build a small three-dimensional atlas that maps the neurons within a developing fruit fly brain. The new digital reconstruction of neurons is 17 times more reliable than those generated using commercially available software, according to Hanchuan Peng, a senior computer scientist at Janelia Farm. Peng is the senior author of a research article describing V3D that was published March 14, 2010, in the journal Nature Biotechnology.

Although the fruit fly brain is only about one-third of a millimeter in diameter, an image that is detailed enough to visualize individual neurons and their connections can only capture a fraction of the structure. Consequently, researchers must take many snapshots of neurons and reassemble them to produce a more panoramic picture.

Developing computational tools to facilitate analysis of massive amounts of biological images and their associated data is one of Peng’s most important goals. "Biologists want to be able to analyze meaningful patterns in the many images they generate from varying experimental conditions, but are unable to do so manually," Peng says. "We take advantage of advanced computing technologies to help them," he said.

In 2006, Peng arrived at Janelia Farm to begin compiling thousands of 3D images of fruit fly brains taken with confocal microscopes. His ultimate goal was to build a comprehensive atlas of the fly's brain. But each image was large—between 500 and 3,000 megabytes of data—and took too long to display on the computer screen. As the number of images in Peng’s database neared the thousands, he turned to commercial and open source tools to help manage the information avalanche. None of those solutions worked, so Peng, who was trained as a computer scientist, decided to start building a tool from scratch, with help from Janelia Farm group leaders Eugene Myers and Julie Simpson, and Gerald Rubin, Janelia Farm’s director and a noted fruit fly biologist.

The team wrote algorithms to speed up the display, or rendering, of the images on the screen. Surface rendering is commonly applied to computer graphics, and is a technique used in video games and motion pictures, such as Avatar. Imaging 3D volumes rather than depicting only objects’ surfaces requires volumetric rendering, which Peng says has been very challenging computationally.

From the start, Peng and his colleagues viewed their software as a kind of digital “Swiss Army knife,” containing critical tools that are easy to use. For example, V3D allows one to drag and drop the images to be analyzed, and to pinpoint a location in a 3D image with a simple mouse click, without the need for stereo-view or virtual reality equipment. "Since we have a very fast renderer for 3D images, we were able to design new approaches to manipulate very large images freely in real time," Peng says.

The software includes a component called V3D-Neuron, which lets users reconstruct and take refined measurements of images of neurons. Users can mouse over and click on any location within an image overlaid with the reconstructed neurons. The display provides crucial information, such as digitized morphology of a neuron and its features, such as the number and length of branches. "You used to have to do that slice by slice, but now you don't need to," he adds.

Early last year, Peng made V3D available on his lab's website and quickly received positive feedback and requests from several groups to collaborate. Now, at least 100 labs are using the tool, he estimates.

Peng's group is working to further improve V3D’s usability and to bundle it with other imaging software tools. Ultimately, he hopes to assemble a 'smart microscope,' by connecting the software to a microscope directly so that a user can acquire an image, manipulate it in real time, and provide feedback to the microscope immediately. "So far, to acquire images, people look through the microscope's eyepiece and decide where to scan," he says. "Then they get the image and do the analysis offline. My dream is to design a system that can acquire the image and do the analysis on the fly. That was not possible before because the image manipulation speed has been too slow."

Peng says he will continue to improve on the 3D atlas of the fruit fly brain, and also plans to collaborate with researchers working on worms and mice. "I'm quite excited about that and I will be able to think more widely about more biological problems now," he says. "People have given good comments about the tool. I definitely think that many more people are going to use it."