NIH researchers put complex, high-resolution data within reach for many more scientists
Scientists have developed new image processing techniques for microscopes that can reduce post-processing time up to several thousand-fold. The researchers are from the National Institutes of Health with collaborators at the University of Chicago and Zhejiang University, China.
In a paper published in Nature Biotechnology, Hari Shroff, Ph.D., chief of laboratory on High Resolution Optical Imaging at the National Institute of Biomedical Imaging and Bioengineering (NIBIB), describes new techniques that can significantly reduce the time needed to process the highly complex images that are created by the most cutting-edge microscopes. Such microscopes are often used to capture blood and brain cells moving through fish, visualize the neural development of worm embryos, and pinpoint individual organelles within entire organs.
As microscopes continue to get better, creating higher resolution images faster, researchers are finding they have more data than time to process it. While the videos themselves can be captured in minutes, the images could be terabytes in size and require weeks or, in some cases, months of processing time to be useable.
One reason it takes so long is that the videos often capture tiny objects that are blurred by the microscope. Such blurring can be reduced by a procedure called deconvolution, but this procedure requires a lot of computing power and time.
A second issue is that many of the microscopes in use today take multiple views of the same organism or cell. Those images need to be positioned correctly and then combined to make 3D images and video. Creating high resolution images from the raw data takes a significant amount of computer processing. And so, while the microscopes have developed to provide researchers with increasingly complex, high resolution images, computing power has limited what techniques are practical for researchers to use—since they know that the majority of the data they collect will go unused.
The first thing Shroff’s lab and his collaborators attempted to do was modify the deconvolution algorithm that is used by many researchers, so it would run faster. This approach was originally proposed for other areas of medical imaging such as computed tomography (CT); however, this is the first time it was successfully adapted for use with fluorescence microscopy. Fluorescence microscopy uses dyes to improve contrast in the specimen, allowing researchers to focus on specific parts of a sample and see how different elements interact with each other.
Second, they reduced the time needed to position and stitch together multiple views of a sample. A key part of this advance relied on a process called parallelization. It is an approach that is sometimes used in supercomputing where instead of processing each individual function one after another, the job is broken up into smaller tasks that can be analyzed concurrently. It is like asking thousands of people to each solve one math problem simultaneously instead of asking one person to solve thousands of problems.
Finally, the researchers showed that they could further reduce the time it takes to process the data by using a neural network, a kind of artificial intelligence (AI). AI is increasingly being used to assist imaging processing and diagnoses. In this case, Shroff and his team trained the neural network to produce cleaner and higher resolution images much more quickly than would be possible otherwise.
“Acquiring modern imaging data is a bit like drinking from a firehose,” said Shroff. “These methods help us obtain valuable biological information faster, which is essential, given the massive amount of data that can be produced by these microscopes.”
These advances expand the use of existing technology, including allowing for imaging of thick samples that produce huge amounts of image data when examined with fluorescence microscopes. The advances are also essential for the use of a growing number of ‘computational microscopes’ in which the post-processing of unintelligible raw data is an essential step in producing the final high-resolution image. Shroff and his collaborators hope that they will help researchers with approaches they would not have thought to try, due to how labor intensive it would otherwise be to create meaningful images.
Scientists from the National Institute of Allergy and Infectious Diseases, the National Institute of Mental Health, the National Institute of Arthritis and Musculoskeletal and Skin Diseases, the Eunice Kennedy Shriver National Institute of Child Health and Human Development, the National Cancer Institute, Harvard University, Sloan Kettering Institute, Yale University, University of Maryland, and the University of Connecticut also contributed to this research.
Rapid image deconvolution and multiview fusion for optical microscopy. Guo M, Li Y, Su Y, Lambert T, Nogare DD, Moyle MW, Duncan LH, Ikegami R, Santella A, Rey-Suarez I, Green D, Beiriger A, Chen J, Vishwasrao H, Ganesan S, Prince V, Waters JC, Annunziata CM, Hafner M, Mohler WA, Chitnis AB, Upadhyaya A, Usdin TB, Bao Z, Colón-Ramos D, La Riviere P, Liu H, Wu Y, Shroff H.Guo M, et al. Nat Biotechnol. 2020 Jun 29. doi: 10.1038/s41587-020-0560-x. PMID: 32601431.
###
NIBIB’s mission is to improve health by leading the development and accelerating the application of biomedical technologies. The Institute is committed to integrating the physical and engineering sciences with the life sciences to advance basic research and medical care. NIBIB supports emerging technology research and development within its internal laboratories and through grants, collaborations, and training. More information is available at the NIBIB website: https://www.nibib.nih.gov.
The National Institutes of Health, the nation's medical research agency, includes 27 Institutes and Centers and is a component of the U.S. Department of Health and Human Services. NIH is the primary federal agency conducting and supporting basic, clinical, and translational medical research, and is investigating the causes, treatments, and cures for both common and rare diseases. For more information about NIH and its programs, visit https://www.nih.gov.