Virtual nanoscopy: Generation of ultra-large high resolution electron microscopy maps

https://doi.org/10.1083/jcb.201201140.dv

Despite the tremendous progress in cellular and molecular biology, there is still a gap in our understanding of many fundamental cellular events and processes. The dynamic behavior of labeled proteins can be studied with fluorescence microscopy; however, despite significant improvements in the spatial and temporal resolution of optical microscopy, other techniques are required to understand their structures, structural state, and ultrastructural context. Transmission electron microscopy (TEM) is able to bridge this resolution gap as it allows rapid and smooth switching between imaging of structures varying in size by three orders of magnitude. At low magnifications, overview images of the specimen can be recorded, albeit with the relatively low resolution that relates to the size of the detector pixel elements. The resolvable details that can be obtained by TEM from biological material are in most cases 2 nm or less. At these higher magnifications, the size of the detector restricts the field of view and hampers the recording of significantly large areas of cellular structures with the required small sampling distance. The traditional workflow of the electron microscopist is to interactively screen the specimen and take overview images at relatively low magnifications and to record several snapshots of regions of interest at higher magnifications. This interactive approach can introduce bias toward the outcome of the experiment because only those details are recorded that were judged to be relevant at the time of imaging. Recording additional snapshots at a later time point is cumbersome, in particular for cryo-vitrified samples that are sensitive to deteriorating effects such as electron dose or contamination.

Here, we describe a methodology to provide unbiased high resolution data access while maintaining the lower resolution overview of the cellular context. In our approach the region of interest is recorded as a sequence of partially overlapping, high resolution TEM images that are combined (stitched) into large, two-dimensional overviews (virtual slides). Smaller virtual slides, consisting of a few tens of TEM images, have been successfully collected before and stitched together with panoramic photography programs such as PanoTools, AutoStitch, Hugin, or Adobe Photoshop. However, for large cellular areas, tissues, or even whole model organisms, the task of data collection and stitching is daunting and calls for an automated pipeline. Technical hurdles to be overcome relate to maintaining specimen focus, image collection speed, robust stitching of low signal-to-noise images, correcting for instability in the electron beam intensity and detector gain, image distortions, and idiosyncrasies in TEM specimen stage movements (Pulokas et al., 1999). The stitching algorithm should scale well with the number of acquired images.

Our approach is conceptually similar to virtual microscopy where digital slides are inspected on a local computer or remotely through the network, allowing the user to zoom in and out on any portion of the entire sample as if they were operating the microscope. Our method is optimized for the specific requirements of TEM with regard to sample preparation, data collection, stitching, and data mining.

Recently, the application of large-scale EM for the ultrastructural mapping of resin-embedded neural systems (Anderson et al., 2009) on a custom-made electron microscope was described (Bock et al., 2011). Our method for collecting “virtual nanoscopy” slides is an improvement over this approach, as it is suitable for standard TEMs, allows for parallel data acquisition, can yield extremely large slides, and is applicable to a large variety of biological samples, including resin-embedded, immunogold-labeled, and unstained cryo-vitrified samples. Virtual nanoscopy does not suffer from sparse or possibly biased selection of regions of interest for high resolution imaging and provides an objective and representative approach to record, communicate, and share data of large areas of biological specimens at nanometer resolution.

Here, we first describe the pipeline for the generation of ultra-large, high resolution electron microscopy maps. We highlight the novel aspects as well as the limitations of our tools. Subsequently, we illustrate the methodological significance of virtual nanoscopy for four different systems; mouse glomeruli, human dendritic cells, mouse embryonic fibroblasts, and zebrafish.

Virtual nanoscopy requires high quality samples. When relevant, chemical fixation protocols were optimized for each specimen type to ensure proper ultrastructural preservation over the entire region of interest. Slot grids were used to provide the largest possible free viewing area. Only grids with an impeccable quality of the support film were used to minimize the risk of film rupture during data collection. A continuous carbon layer was applied to the support film to strengthen it and to reduce charging artifacts.

A software module called MyTEM was developed to control FEI transmission electron microscopes through the TEM Scripting interface. It allows for parallel data acquisition where the specimen is translated during detector readout. With this approach we record up to 10,000 2k × 2k slow-scan CCD images per day. Along with each image, MyTEM collects and archives metadata on the microscope status. MyMesh is the user interface that submits calls to MyTEM and interacts with the microscope. The user can request the collection of a queue of square, rectangular, or convex polygonal virtual slides and define imaging parameters such as the required defocus, stage settling time, overlap, and detector binning. The module MyStitch combines (stitches) the TEM images into a virtual slide, correcting for inaccuracies in specimen positioning and variations in beam intensity and detector gain (see Materials and methods). MyStitch can stitch the images “on the fly,” i.e., during the data acquisition process. The virtual slides are written as a multi-resolution image pyramid to allow for rapid visualization, among others, with our Java viewer MyView.

Virtual slides are routinely collected in our laboratory from a wide range of samples, including resin-embedded cells, tissue, biopsies, negative-stained purified macromolecular complexes, and cryo-sections, as well as intact vitrified cells grown on EM grids. Virtual slides of up to 281 gigapixels (Gpixel) were generated. The time required for data collection scaled linearly with the amount collected and lasted up to 4.5 d for the 281-Gpixel slide. On average, ∼15 s per 4k × 4k image was needed, including (pre) exposure time, readout time, stage settling time, and overhead such as focusing.

All data were collected with a serpentine motion of the stage to prevent large stage movements between successive frames. A pragmatic trade-off was made between stage positioning accuracy and the time required for the stage to settle at the next imaging position. A small stage-dependent backlash correction was applied by consequently approaching the target position from the same direction. The stage positioning accuracy (3σ) for our largest dataset was 170 nm in X and 60 nm in Y for the movements along the long side of the serpentine pattern. For the orthogonal direction, the accuracy was 460 nm in X and 500 nm in Y. An overlap of 20% between frames was chosen to overcome a large systematic offset error of 900 nm between successively scanned rows, although alternative methods exist (Pulokas et al., 1999).

Even though several image-stitching algorithms can incorporate knowledge about the data collection pattern or even explicit sample positions, such information is commonly discarded after the first alignment. TEM data collection imposes tight constraints on the relative positions of the individual images. In principle, all images are recorded with an identical magnification and should show little, if any, rotations as the sample is effectively moved over a Cartesian grid. Although corrections for scale, rotation, and projective distortions can produce very smooth virtual slides, unrestrained refinement of such parameters is unlikely to preserve the true geometry of the sample. Therefore, we designed a scheme with a tight integration of the metadata and a minimal number of parameters to be refined. Such a scheme tolerates local misalignments without compromising the global geometry.

Our scheme is based on a weighted least squares solution minimizing the difference between measured and refined displacement vectors between overlapping images. The relative positions of the individual TEM images within the final virtual slide are then calculated (Eq. 1, Materials and methods). The weighted root mean square deviation (rmsd) between the measured and the final displacement vectors forms the residual error. Typical errors ranged from 1–15 pixels. Featureless image regions can result in inaccurately measured displacement vectors that will deviate from the final displacement vectors. The robustness of the algorithm originates from outlier rejection, prior knowledge, and an empirical weighting scheme (see Materials and methods). The required memory for the robust least squares method scales with the number of frames acquired and remains less than 16 Gb for the largest virtual slide shown.

The intensity and position of the illuminating electron beam might vary during data collection. Normally, the integrated intensity fluctuates less than a few percent for typical exposure times on the order of a second. We observed larger variations in intensity in some of our datasets that need to be corrected for. Non-uniform beam illumination or a suboptimal flat-field correction of the detector would introduce slight ramps in the individual frames. These would result in an evident sawtooth intensity pattern within the virtual slide. Our algorithm applies average intensity as well as low-order polynomial gain corrections to each frame. The algorithm takes the median intensities of subregions in overlapping areas as input and minimizes the least square difference in these (Eq. 2, Materials and methods). This system of equations is highly overdetermined, allowing for random selection of subregions within an iterative scheme. Such a scheme is less prone to specific outliers. A typical resulting intensity rmsd before and after scaling is 353 and 112 ADU (analogue to digital unit) for one of the datasets presented here that has a median intensity of 18,000 ADU.

The virtual slides are written as image pyramids (see Materials and methods). The time required to write these is mainly determined by disk speed and proportional to the number of images being read. Arbitrarily sized virtual slides can be visualized interactively provided that data access overhead is carefully managed.

We combined virtual TEM and light microscopy to study the role of the endothelial glycocalyx in the filtration barrier. All cells—from single cell microorganisms to highly organized mammalian cells—are shielded from their surroundings by the glycocalyx, a membranous, carbohydrate-rich layer. The endothelial glycocalyx protects the vascular wall from direct exposure to flowing blood, contributes to the vascular permeability barrier and its anti-adhesive properties, and stimulates the endothelial release of nitric oxide in response to fluid shear stress (van den Berg et al., 2007). Proteinuria, which results in excessive levels of protein in urine, is a hallmark of kidney diseases (Haraldsson and Jeansson, 2009). The glomerular filtration barrier consists of podocytes, glomerular basement membrane, and endothelial layer (Pavenstädt et al., 2003; Haraldsson et al., 2008). The exact role of various components, in particular the luminal endothelial surface glycocalyx, is still controversial. It is hard to study each component of the filtration barrier separately as they all influence each other, and it is thus imperative to characterize the glomerular system as a whole. However, quantitative studies such as the EM examination of individual ferritin particles passing the filtration barrier (Jarad et al., 2006; Abrahamson et al., 2007) are limited to using high resolution snapshots and do not address the interplay of cell types over large scales.

We took advantage of our ability to acquire high resolution information for the full organ to assess the role of the glycocalyx in the renal glomerular barrier. To this end, the passage of endogenous albumin was determined in 16-wk-old mice treated intravenously for 4 wk with active or inactivated (control) hyaluronidase, which resulted in a 50% glycocalyx volume reduction without any gross changes in renal morphology (Meuwese et al., 2010). Kidney sections were labeled with anti–mouse albumin antibody coupled to horseradish peroxidase and incubated with DAB (see Materials and methods). The DAB precipitate was visualized both with reflection contrast microscopy (RCM) and TEM. RCM allows for the screening of large numbers of glomeruli from kidneys from different mice, whereas TEM enables to localize the DAB precipitation at high resolution. Positive RCM images reveal reflective material outside the vascular lumen that may be located at the podocytes and the boundaries of their processes (Fig. 1). To unambiguously correlate the observed extravascular reflection with specific DAB precipitation, the positive RCM image and the TEM virtual slide were overlayed (Fig. 2). Within this overlay it was possible to visualize, at high resolution, the specific DAB precipitation of the reflection found in RCM. The TEM images provided the resolution to localize albumin at the surface of the podocytic foot processes, indicating that modest changes in the endothelial glycocalyx content could result in albumin leakage over the glomerular filtration barrier (Fig. 2).

Virtual nanoscopy can be combined with immunolabeling. Immunogold labeling on thin cryosections of chemically fixed cells (Tokuyasu, 1980) has become a standard method for immuno-EM. Most commonly, colloidal gold particles are attached to secondary antibodies, which are used to localize primary antibodies designed to bind a specific antigen. Consequently, gold particles can be 15–30 nm away from the site to which the primary antibody is bound. The use of large gold particles facilitates the identification of these particles at lower magnification; however, it also further increases the uncertainty about the precise location of the targeted molecule. For presentation purposes, traditionally, one or more figures containing the gold particles within a field of view of a few micrometers square are shown. This introduces an inherent bias in the selection of the area of interest and omits information about heterogeneity in labeling efficiency and nonspecific background labeling, as well as variety in the cellular position of the targeted molecule.

These limitations can be overcome by virtual nanoscopy. We applied the method to immunogold-labeled sections of human dendritic cells loaded with oncolytic viruses. These cells are being tested for the delivery of reovirus to melanoma cells in the presence of neutralizing serum (Ilett et al., 2011). It is hoped that dendritic cells can be used for viral delivery to tumors and the induction of an antitumor immune response. Ultrathin cryosections of reovirus-infected human dendritic cells were incubated with anti-reovirus σ3, a bridging rabbit anti–mouse IgG antibody, and protein A–gold particles 10 nm in diameter (Ilett et al., 2011). Fig. 3 A shows a section that includes ∼84 cells. To visualize the gold labels at low magnification over the entire area, a post-processing step was applied to localize the gold particles and to highlight them by semi-transparent coloring (see Materials and methods). High concentrations of label became visible as green patches at low magnification (Fig. 3 A). The viral capsid protein could be detected both at the cell surface (Fig. 3 B), leaving it accessible to neutralization, and within the dendritic cell (Fig. 3 C), protecting it from an immune response (Ilett et al., 2011). The information of occurrence of an event could become lost during manual imaging of an immunogold-labeled section. Virtual nanoscopy combined with post-processing permits the visualization of both the specificity as well as the incidence of certain events within a slide.

Cryo-electron tomography of unstained vitrified cells has the promise to provide structural insight of supramolecular machineries within their native crowded cellular environment. It was postulated that pattern recognition methods, such as template matching, will be able to identify and localize macromolecules within a cell, providing a visual approach to proteomics (Nickell et al., 2006). The specimen would need to be sufficiently thin (<500 nm) to be examined with intermediate voltage TEMs. Cryo-ultramicrotomy (McDowall et al., 1983) and, more recently, focused ion beam micromachining (Rigort et al., 2012) are methods for producing thin frozen-hydrated sections of cells and tissues. We examined mouse embryonic fibroblasts (MEFs) that were grown directly on carbon supported by EM grids. These cells can stretch out such that they provide vast suitable thin regions (Koning et al., 2008; Koning, 2010). They were vitrified by plunge-freezing (see Materials and methods). Fig. 4 shows a virtual slide of such region. A variety of cellular structures could be recognized, such as actin filaments, coated vesicles, microsomes (Fig. 4 C), mitochondria (Fig. 4 D), multilamellar bodies, and microtubules (Fig. 4 E). This virtual slide illustrates the feasibility to perform automated stitching on frozen hydrated samples. The automated data collection scheme allowed for a careful control of specimen exposure, which is mandatory as the ionizing electron beam will cause radiation damage. The stitching was done at full resolution of the individual TEM images, exploiting the fact that supramolecular features only become visible at high magnification. Alternative multi-resolution registration schemes are prone to errors due to the unfavorable signal-to-noise ratios in these data.

Over the last two decades, zebrafish (Danio rerio) has become a powerful vertebrate model system to investigate human disease and development. Zebrafish are small (growing up to 4 cm in captivity), inexpensive to maintain in large numbers, and easy to handle during experimentation. A major advantage of zebrafish is that their embryos are optical transparent, allowing for sophisticated large-scale imaging studies.

We demonstrate the scalability of our method by the collection of a virtual slide of a complete zebrafish sagittal section (Fig. 5). A 5-dpf embryo section was imaged at 1.6-nm pixel resolution over an area of 1.5 × 0.6 mm², yielding a net total of 281 Gpixel (see Materials and methods). At low magnification, organs such as the eye, olfactory pit, mid- and hindbrain, liver, cartilage, intestine, pancreas, swim bladder, notochord, and muscles could be recognized (Fig. 5 A). Fig. 5, B–E, shows high magnification snapshots displaying microvilli in the intestine (B), cross sections of cilia of the olfactory pit (C), notochord caveolae (D), and a junctional complex with two desmosomes (E). We examined the virtual slide for mitochondrial membrane wrapping and morphological evidence of macroautophagy and microautophagy.

Autophagy is the major self-degradative process in eukaryotic cells (Mizushima and Komatsu, 2011). It has been suggested to play a protective role during aging, defense against pathogens, cell death, lysosomal storage diseases, neurogenerative diseases, and tumorigenesis (Todde et al., 2009; Mizushima and Komatsu, 2011), as well as roles in embryonic development, reticulocyte (Ney, 2011), megakaryocyte maturation (Colosetti et al., 2009), and cellular differentiation. Autophagy processes are subdivided into roughly three classes: (1) macroautophagy, which is mediated by autophagosomal fusion with lysosomes (Juhasz and Neufeld, 2006; Mizushima and Komatsu, 2011); (2) microautophagy (Sakai and Ogawa, 1982), which is mediated by lysosomal protrusions; and (3) chaperone-mediated autophagy (Mizushima and Komatsu, 2011). The origin of the autophagic membrane has long been an open question in cell biology (Juhasz and Neufeld, 2006; Simonsen and Stenmark, 2008; McEwan and Dikic, 2010), with both omegasome structures on rER (Axe et al., 2008) and the outer membrane of mitochondria (Hailey et al., 2010) implicated as the source of autophagosomal membranes.

The zebrafish virtual slide was surveyed for membrane wrapping of mitochondria and potential evidence of macro- and microautophagosomal processes within this developing embryo for all tissues simultaneously. ER wrapping of mitochondria was also recorded, although it is reported to be distinct from autophagy (Eskelinen, 2008). It is thought to play a role in calcium transport from the ER to mitochondria as well as apoptosis (Rizzuto et al., 2004; Walter and Hajnóczky, 2005; Szabadkai et al., 2006).

Mitochondria within the brain, eye, inner ear, muscle, and olfactory pit rarely showed obvious membrane structures located in their direct vicinity. Muscle mitochondria were in general tightly positioned between myofibrils. Some of these mitochondria were located just next to a triad, where depolarization of the T-tubulus results in Ca²⁺ release from the terminal cisterna of the sarcoplasmic reticulum.

Mitochondria varied in size among the different tissues, with the smallest found within the brain and the largest within the muscles. (This is best seen interactively by zooming in and out on the online version of Fig. 5). The exocrine acinar cells of the pancreas contained mitochondria that were tightly packed within the prominent rough endoplasmic reticulum (rER); however, individual rER cisternae were not obviously aligned with the mitochondrial membrane. There were a number of places within the sagittal slide where one could see eosinophil granulocytes, white blood cells with characteristic primary lysosomes (diameter 0.5–1.5 µm) containing major basic protein crystals (the lattice spacing can be seen in some of these crystals). Their mitochondria were located nearby rER that was less densely packed compared with the acinar cells. No obvious membrane wrapping was observed for the mitochondria of the eosinophils.

Mitochondria within the pronephric duct had very pronounced membrane structures around them that were anchored in the basal membrane of the duct. These structures are basal infoldings that play a role in the solute recovery of blood filtrate produced by the embryonic glomeruli.

The mitochondria of the notochord as well as the intestine could be grouped into two classes (Table 1). Within the notochord, both bare mitochondria and mitochondria in close contact to ER cisternae could be found, the first mainly at the apical side of the notochordal epithelium (shown in green in Fig. 6), the latter at the basal side (shown in red in Fig. 6). A similar division is observed for the mitochondria of the intestinal columnar epithelial cells (Table 1, Fig. 6). The majority of mitochondria in enterocytes were located at the apical side, just underneath the striated border. These mitochondria were in close proximity to rough and smooth ER, though no obvious wrapping was observed. Mitochondria at the basal side of the enterocytes were frequently lined by smooth membrane structures.

Many of the cartilage cells showed signs of hypertrophy: these enlarged chondrocytes showed cytoplasmic and nuclear swelling, and the ER cisternae were dilated. A total of 124 chondrocytes and 100 distinct mitochondria were counted. 80% of these mitochondria were partly wrapped by rER (Table 1). The liver and the yolk region showed an abundance of mitochondria with long ER cisternae wrapped around them (Fig. 6).

The main panel of Fig. 7 shows a close-up view of the 5-dpf sagittal zebrafish section in which the liver is highlighted. At low magnification, mitochondria already appear differently compared with mitochondria in most other tissues: they were large and numerous, some appeared destructed, and several of them had large irregular shapes suggesting fusion of individual mitochondria. Cross sections through Disse’s space were observed with similar dimensions as the cross sections through mitochondria. A total of 433 hepatocyte mitochondria and 23 cross sections of Disse’s space were counted. Most of the hepatocyte mitochondria were wrapped by rER cisternae (Table 1; Fig. 7, A–D). About one quarter (n = 99) of the mitochondria had one or more lysosomes nearby (Fig. 7 C) or in intimate contact with the mitochondrion (Fig. 7, E–H). During autophagy, autolysosomes are formed by the fusion of a lysosome with the double membrane autophagic vacuoles (Mizushima and Komatsu, 2011). The structures we observed within the liver are different, as lysosomes fuse with mitochondria before smooth membrane enclosure. Instead, rER cisternae are found wrapping both mitochondria and their (partly) fused lysosomes. Some of these cisternae (colored orange in Fig. 7) have a significantly smaller number of ribosomes.

Unlike hematoxylin & eosin histological slides (Penn State Zebrafish Atlas; http://zfatlas.psu.edu), where preparation artifacts obstruct a morphological investigation of the yolk, the yolk region was well preserved for the data presented here. Similar to the liver, extensive rER wrapping of mitochondria was observed within the yolk region (this region is delineated by the yolk sac membrane and includes the yolk syncytial layer). The region could be divided into different stages of the resorption process; the large lipid droplets in the center (Fig. 8, right side) become absorbed later compared with the thinner periphery (Fig. 8; left side). Mitochondria within the central area displayed very pronounced wrapping of individual rER cisternae around them (Fig. 8 J). Individual cristae could be observed in most of these mitochondria, and lysosomes could be observed nearby. Mitochondria farther away from the large lipid droplets were partly wrapped by rER, were swollen, had loose internal structure, and were more circular (Fig 8 I). Even further from the central part of the yolk (e.g., position H in Fig. 8), the enlarged mitochondria were less frequently wrapped by rER and showed an amorphous structure inside that lacked cristae. Obvious examples of autophagosomes were found within the yolk region of the zebrafish embryo (Fig. 8, A, B, F, and G). Furthermore, we observed numerous vesicles with arm-like protrusions of varying length and curvature within this region (examples are shown in Fig. 8, C–E), indicative for microautophagy.

Large-scale EM is an emerging approach for collection of data from significantly larger areas of samples than was previously possible (Anderson et al., 2009; Kaynig et al., 2010; Bock et al., 2011). Serial section EM has recently been used to investigate structure–function relationships of a cohort of cells of the mouse primary visual cortex that encompasses millions of cubic micrometers (Bock et al., 2011). EM is the optimal tool for ultrastructural studies of cells, but visualization of large areas such as organs, tissues, or even entire animals at nanometer resolution has been challenging. Different solutions have been proposed to mitigate this problem, such as the use of multiple microscopes in parallel (Anderson et al., 2009) or the custom-made construction of a TEM camera array (Bock et al., 2011). With the latter, more than a thousand mosaics of almost 10 Gpixel each were collected. Examples of tools that have been developed to align sequential slides include work on 2D LM slides (array tomography; Micheva et al., 2010), TEM slides (serial section TEM; Anderson et al., 2009; Akselrod-Ballin et al., 2011), and tomograms (serial section electron tomography; Soto et al., 1994). A volumetric sampling of an entire 5-dpf zebrafish embryo would require thousands of virtual slides like the one presented in Fig. 5. Although developments in sample preparation, imaging, and image processing technologies might make such an approach possible in a not-too-distant future, the time required to segment, annotate, and interpret these data are likely to remain a limiting factor.

For the near future, we foresee an increasing need for the routine collection of large 2D TEM slides, in particular if these can be collected on standard microscopes. Our tools are geared toward the acquisition and visualization of large 2D slides. The explicit collection and use of metadata allows for a global robust refinement of the relative positions of the individual TEM images within a final virtual slide while preserving the geometry of the sample. There are several panoramic photography programs available that are capable of generating large maps. However, these suffer from various limitations that make them less suitable for the diversity and sheer size of the data presented here. For example, Image Composite Editor (ICE; Microsoft Research) does support “structured panoramas” and automatically detects the serpentine order in which our data have been collected. Unfortunately, however, this information is discarded in subsequent steps: the program allows individual images to become freely repositioned. The simplest camera motion option the program offers refines translation, scaling, and rotation of the images. Such a scheme can result in major deformations. MosaicJ (Thévenaz and Unser, 2007) can optionally make use of knowledge about the path of the scan. Their Java implementation as a plugin for ImageJ (National Institutes of Health, Bethesda, MD) effectively limits the size of the final slide to be smaller than the RAM memory of the computer. The computationally efficient algorithm pays special attention to ensure the highest registration accuracy. Similar to ICE, prior positional information is discarded in this step, which can easily result in large global geometric deformations. The package ir-tools (NCR Toolset, University of Utah, Salt Lake City, UT) can deal with manual TEM image acquisition on information-dense sections of neuronal tissue. It includes a brute force component (ir-fft) for finding neighbors among a given set of images and uses these relations to position the individual images. Such an approach is computationally intensive and vulnerable to errors when the specimen contains large sparse areas (such as the pharynx and swimbladder in the zebrafish; Fig. 5). Our restrained positioning approach will not ensure the highest possible registration accuracy, as illustrated by, e.g., the large residual error for the zebrafish slide (rmsd = 15; see Results). This could form a breakpoint for subsequent processing steps such as automated segmentation. All algorithms have difficulty aligning TEM images that include folds, holes, or gridbars. Interactive panoramic photographic programs such as Hugin or Adobe Photoshop could be used to overcome this when applied to small subsets of the data.

Recent developments in large-area scanning electron microscopy (SEM) of resin-embedded samples complement our efforts. The use of a scanning probe facilitates the collection of individual images that can be as large as one gigapixel each. Such a larger field of view reduces the number of stage movements and increases the stage step size, making the system more robust against stage idiosyncrasies. The resulting virtual slides will have a smaller number of “seams,” but, currently, such seams are clearly visible due to problems with preirradiation, focusing, and distortions. Unlike SEM, where only the surface of a sample is visualized, TEM can be used to image thicker sections (up to a few hundred nanometers). Virtual TEM slides could, posteriori, be complemented by 3D tomograms of selected regions of interest.

Virtual nanoscopy will help to disseminate ultrastructural TEM data to a broader community because the slides can be viewed independently by a large number of people in diverse locations. Its development was inspired by the imaging requirements for correlative light and electron microscopy (CLEM; Koster and Klumperman, 2003; Giepmans et al., 2005; Sartori et al., 2007; van Rijnsoever et al., 2008; Mironov and Beznoussenko, 2009; Nixon et al., 2009; Plitzko et al., 2009; van Driel et al., 2009; Brown and Verkade, 2010; Kukulski et al., 2011; Watanabe et al., 2011). In CLEM, tagged biological molecules are imaged by light microscopy (LM), and their context is visualized by electron microscopy (EM). Virtual nanoscopy could become a powerful tool in bridging the gap between live-cell imaging and EM (CLEM) because registration allows access to data in both modalities from the same field of view.

Implementation of the methods presented here scales well with the size of the virtual slide collected and is in practice only limited by the acquisition time and the lifetime of the sample. It proved to be robust for a wide range of samples, including stained, unstained, room-temperature, and cryo-samples. Furthermore, it was developed for standard transmission electron microscopes. Post-processing provides clear advantages for immunolabeling and CLEM studies. Combined with other developments toward large-scale TEM, STEM, and SEM, it will increase the value of EM for life sciences.

Vibrotome sections of the right kidney of 16-wk-old mice, chronically infused with either active or inactive hyaluronidase for 4 wk, were fixed overnight in 2% PFA, washed twice with phosphate-buffered saline (PBS), and blocked for 30 min on ice with 10% normal goat serum and 0.3% Triton X-100 in PBS. Next, samples were incubated overnight at 4°C with HRP-conjugated goat anti–mouse albumin antibody (Bethyl Laboratories, Inc.) in 1% normal goat serum in PBS, washed twice with PBS, stained for 30 min at 4°C with DAB and H₂O₂, washed with PBS and 0.1 M sodium cacodylate, incubated for 1 h with 1.5% GA and 1% PFA, rinsed in cacodylate, and postfixed for 1 h in 1% osmium tetroxide and 1.5% potassium ferrocyanide. Samples were dehydrated in a graded ethanol series up to 100% and embedded in Epon. Sequential 100-nm sections were mounted on a copper slot grid covered with formvar support film and a 3-nm carbon coating for TEM, and on a water drop on a clean glass slide for RCM (Prins et al., 2005). The RCM sample was mounted with immersion oil (Immersol 518F) on an RCM-adapted microscope (reflection contrast device RC; Leica). Images were recorded with a 1.25 NA 100× objective.

Immature dendritic cells were derived from monocytes from healthy blood donors using anti-CD14 magnetic beads, and loaded with reovirus type 3 Dearing (Oncolytics Biotech, Inc.) at 4°C for 3 h (Ilett et al., 2011). The infected cells were fixed for 2 h in PHEM buffer (60 mM Pipes, 25 mM Hepes, 10 mM EGTA, and 2 mM MgCl₂) containing 2% PFA and 0.2% GA, pelleted, embedded in 12% gelatin, and cut with a razor blade into cubes of ∼1 mm³. The sample blocks were infiltrated in phosphate buffer containing 2.3 M sucrose for 3 h, mounted on aluminum pins, and plunged into liquid nitrogen. Ultrathin cryosections (Tokuyasu, 1980; Peters et al., 2006) were incubated with anti-reovirus σ3, a bridging rabbit anti–mouse IgG antibody, and protein A–gold particles of a 10-nm diameter (Ilett et al., 2011).

Mouse embryonic fibroblasts (MEFs) were cultured as described previously (Koning et al., 2008; Koning, 2010) and grown on carbon-coated formvar that was supported by gold EM grids. Excess medium was blotted at 37°C and 100% humidity using a Vitrobot Mark IV (FEI Company). The blotted grids were plunged in liquid ethane and stored in liquid nitrogen for further use.

Wild-type (AB) zebrafish (Danio rerio) embryos were raised until 5 dpf, anesthetized, and chemically fixed in either (a) 1.5% glutaraldehyde (GA); (b) a mixture of 1.5% GA and 1% paraformaldehyde (PFA); or (c) a mixture of 1.5% GA and 4% PFA; all were in 0.1 M sodium cacodylate for 1 h, rinsed twice, followed by incubation in 1% osmium tetroxide in 0.1 M sodium cacodylate for 1 h. After rinsing in cacodylate buffer, fish were dehydrated in a graded ethanol series up to 100% and embedded in Epon. Pilot experiments revealed that the morphologies of the fish fixed in 1.5% GA with or without PFA were similar; data collection was performed on a fish fixed in 1.5% GA without addition of PFA. Sagittal sections of 800 nm were stained with toluidine blue and analyzed for general morphological features under the light microscope. 100-nm sections were transferred onto slot grids covered with a formvar and carbon film, and post-stained with uranyl acetate and lead citrate.

Imaging of resin-embedded samples with TEM causes sample shrinkage, in particular in the direction of the beam, resulting in an increased contrast and translucency of the irradiated area of the specimen. As a consequence, a controlled preirradiation of the sample at low magnification is crucial for obtaining optimum results. The data shown in Figs. 2 and 5 were collected with an approximate incident flux density of 0.5 e⁻Å⁻²s⁻¹ and 1 s integration time per image, whereas these samples were uniformly preirradiated at low magnification with 5 e⁻ nm⁻²s⁻¹ for at least 10 min before data collection. The immunogold-labeled sample was preirradiated with a lower incident flux density.

All TEM data were collected on FEI Tecnai microscopes. The RCM images were collected on a Leica DMR microscope equipped with an RCM oil-immersion objective endowed with a quarter lambda plate and a reflection contrast module RC (Prins et al., 2005). The RCM images were recorded at room temperature with a Leica DFC420 C camera and the Leica application suite microscope software.

The virtual slide shown in Fig. 2 was recorded at 80 kV with a magnification at the detector plane of 9460. A total of 990 4k × 4k images were collected with an FEI Eagle CCD camera in a time interval of 240 min. Every 10th image, the FEI autofocus routine was used to maintain the sample at −1 µm defocus. The resulting slide of 172 × 158 µm² consists of 108,544 × 99,840 pixels of 1.6 nm square each.

The virtual slide shown in Fig. 3 was recorded at 120 kV with a magnification at the detector plane of 18,080. A total of 441 4k×4k images were collected in 114 min at −1 µm defocus. The resulting slide of 59 × 59 µm² consists of 71,168 × 70,656 pixels of 0.8 nm square each.

The cryo-TEM slide shown in Fig. 4 was recorded at 120 kV with a magnification at the detector plane of 32,000. A prespecimen shutter was used to avoid unnecessary exposure of the sample to the damaging electron beam. An incident flux of 5 e⁻ Å⁻² s⁻¹ was used and the integration time per image was 1 s. The illuminated specimen area was adjusted such that the beam diameter within the detector plane was less than two times the detector width, thereby limiting the integrated flux for each spot of the specimen to a maximum of 15 e⁻ Å⁻² s⁻¹. The data were recorded with −10 µm defocus as determined from a single spot nearby the imaged region. A total of 117 2k × 2k images were collected in 18 min. The resulting slide of 19 × 13 µm² consists of 20,480 × 13,824 pixels of 0.9 nm square each.

The virtual slide shown in Fig. 5 was recorded at 120 kV with a magnification at the detector plane of 9460. A set of points was manually selected to outline the zebrafish and the convex hull of these points was used to define the data collection area. A total of 26,434 unbinned 4k × 4k images was collected with a FEI Eagle CCD camera (>8 s readout time full frame) in 4.5 d. The sample was maintained at −1 µm defocus throughout the whole data collection. The resulting slide of 1,461 × 604 µm² consists of 921,600 × 380,928 pixels of 1.6 nm square each. The net data content of this slide is 281 Gpixel.

High resolution data acquisition of large fields of view can take many hours, which necessitates overnight unattended data collection. FEI Tecnai TEMs can be equipped with a cold-trap that is liquid nitrogen (LN₂) cooled. The LN₂ is normally placed in an 800-ml capacity Dewar on the side of the TEM column. We designed a support for a 6-liter liquid nitrogen Dewar (thermos) which is sufficient for ∼12 h unattended data acquisition. Such an adaptation is not needed for automated data acquisition on the FEI Polara or FEI Titan series of microscopes.

The program MyStich has been developed to combine (stitch) TEM images produced by MyMesh into a virtual slide. Each image recorded by MyTEM is accompanied by metadata. The stitching program MyStitch uses these metadata to determine overlapping TEM image pairs. It can start the stitching “on the fly” during the data acquisition process. Separate computational jobs are launched for all image pairs to compute image shifts by cross-correlation, normalized cross-correlation coefficients, and intensity scale factors. The submission of separate jobs allows for easy parallelization of the stitching process. The overlapping regions are padded to optimize Fast Fourier transform processing speed as well as to overcome the fourfold ambiguity in cross-correlation peak positions (Mastronarde, 2005).

The relative positions of the TEM images within the virtual slide are determined using a weighted least squares method, based on the calculated image shifts and an empirical weighting scheme with weights w that are a function of the normalised cross-correlation coefficients. Assuming that a square scan of n_x times n_y images was collected, P = 2n_xn_y − n_y − n_x image pairs and q = n_xn_y images are obtained. p is an upper limit for non-square scans. Let N be a p×2 matrix containing the indices of the p neighboring pairs. We now define a p×q matrix M with:

and zero elsewhere. We calculate a least square solution by applying the Moore-Penrose pseudo-inverse (Penrose, 1955) to the overdetermined system of linear equations with $x→$ being the image position vectors and $d→$ being the measured image shifts for each image pair:

where W is the diagonal weighting matrix based on the weights w. The position of one of the images will be fixed to the origin, which slightly modifies M and $x→$⁠.

The weights w are calculated as the normalized cross-correlation coefficients to a power α. Cross-correlations smaller than a certain threshold (τ) were set to that threshold value. This threshold has to be larger than zero to avoid singular matrices in Eq. 1. A value of α larger than one increases the weight on the more reliable image shifts. Our empirical-derived values for α and τ are 5 and 0.15, respectively.

Eq. 1 could result in nonoverlapping image positions, in particular for featureless areas where a false registration can locally dominate W. These outliers are identified and accounted for in an iterative scheme where improbable image shifts are down-weighted. For this purpose, image shifts are clustered according to the last stage movement, and used to calculate median values and spread.

Scaling of the intensities is needed for several reasons. In practice, imaging conditions will suffer to some degree from nonuniform illumination and imperfect detector gain corrections, which introduce clear visible effects such as a sawtooth pattern within the uncorrected image stitch. In addition, the illumination intensity might change during a multi-hour data collection session because of fluctuations in the brightness and alignment of the electron gun. In most cases, these fluctuations will be gradual, though a sudden change in brightness is possible when the gun experiences instability. It is also possible that the characteristics of the digital image detector (bias, gain) change during data collection. Another possibility is that the preirradiation step of resin-embedded specimens was too short or nonuniform.

Because of these effects, we correct for intensity and gain in a weighted least squares scheme by minimizing:

where S are scale factors for the individual images, I sampled intensity values, and G is a gain image described as a third-order polynomial. We normalize the gain such that

A variable number of gain images can be used. Eq. (1) has a trivial solution for S = 0. To overcome this, we add a prior term to ensure that the scale factors are close to unity (Brown and Lowe, 2007):

The parameters σ_I and σ_s are the estimated standard deviations of the scaled intensity differences and scale factors, respectively, and can be tuned if necessary. The intensity values I are sampled for a number of subareas (typical size: 64 × 64 pixels) of the overlapping area between each image pair.

The virtual slides are written as tiled image pyramids, consisting of a series of layers where each successive layer is binned with a factor of two. All layers consist of tiles of 256 × 256 pixels in size. The base layer is compiled from the original images, after applying intensity scaling and gain corrections. We consecutively load the original images and map the content to the respective tiles. The tiles are written to disk as soon as they receive all contributions from the overlapping original images. This scheme ensures that the original data are read only once while keeping the memory requirements low.

The performance of our pipeline has been evaluated for a wide range of biological samples, which challenges the robustness of stitching algorithms. Panoramic photographer stitching programs such as Microsoft Image Composite Editor use feature-based registration methods (Brown and Lowe, 2007). In our experience, such algorithms do not perform well for, e.g., empty areas between cells where few or no features are found, or for opaque areas such as nuclei or grid bars. Evaluations of these programs on our data resulted in compositions of shattered fragments and unnecessarily long computation times for ill-defined straightening procedures (Brown and Lowe, 2007).

The majority of the 10-nm-diameter gold particles within the virtual slide shown in Fig. 3 were localized with matlab (www.mathworks.com) and diplib (www.diplib.org) routines using criteria such as shape, size, relative intensity, and intensity variation. Thresholds were chosen such that the number of false positives was minimal. For each gold position, a green Gaussian profile with 235 nm full-width-half-maximum was added to the original data, facilitating gold-clustering localization at low magnification while preserving high magnification image content.

The image processing system VIPS (www.vips.ecs.soton.ac.uk) can be used to extract layers from the TEM image pyramid. By selecting a layer of similar physical dimensions as the light microscopy image, a correlation can be made with a variety of programs (e.g., Adobe Photoshop, Hugin). We used matlab routines to determine and correct for the nonrigid transformation between the light and the TEM image.

We use libtiff 4.0 (http://www.remotesensing.org/libtiff) to overcome the 4-Gb file size limit that is characteristic of many other data formats. The image pyramids can be visualized locally and remotely with our platform-independent Java viewer MyStitch. Currently, we use ImageScope (www.aperio.com) for annotations and IIPImage (http://iipimage.sourceforge.net) to serve the virtual slides on the web. The programs MyTem, MyMesh, MyStitch, and MyView are available from the authors upon request.

All internal as well as external users are acknowledged for their constructive feedback. We thank Max Otten and Remco Schoenmaker (FEI company) for help with TEM Scripting; Frans Prins (Department of Pathology, LUMC) for valuable help with RCM; Ronald Limpens and Montserrat Bárcena (Department Molecular Cell Biology, LUMC) for the immunogold-labeled section; Kasia Mościcka (Department Molecular Cell Biology, LUMC) for cryo-EM preparation of MEF cells; Malgorzata Wiweger (Department of Pathology, LUMC) for zebrafish embryos; and Elspeth Garman (University of Oxford), Ton Raap, and Hans Tanke (Department Molecular Cell Biology, LUMC) for critical reading of the manuscript.

M.C. Avramut and R.B.G. Ravelli acknowledge financial support from NWO under project number 016.072.321. B.M. van den Berg acknowledges financial support from the Dutch Kidney Foundation (grant CP09.03).

The authors declare no competing financial interests.