Figure 2.
Coordinate system mapping and automatic detection for the correlation strategy. (a) Cell of interest selected using fluorescence microscopy by scanning low magnification images (first and second image). In our experiments, we targeted the Golgi apparatus center of mass (a, third image, white cross). The image position is translated to stage position coordinates and stored in the “LM targets list” (green). (b) Simultaneously, reflected light images (b, first image) are stored, and later used to extract the stage coordinates of landmarks (LM landmarks list, pink). The image is analyzed and a line detector is applied (red lines). The intersection of the lines is used to find grid bar crossings (b, second image including inset). The corresponding detected edges are converted to lines that automatically mark 4 points (b, second image, red dots). Those points are used to determine the center point (second image, yellow dot), and they will be part of the “LM landmarks list.” By convention, the top left corner (yellow arrowhead) is named by associating its unique center point (yellow dot) with the alphanumeric identifier imprinted onto the glass dish bottom. To identify the alphanumeric character on the image, the reflected light image is automatically thresholded and cleaned (b, third right image) using a combination of traditional image analysis pipelines (see Fig. S1) and then passed through a convolutional neural network for classification, in this case, 8Q. (c) In the FIB-SEM, the strategy of mapping is repeated: scan images are taken by the Navigator module (c, first image), and the grid bar crossings are detected to calculate the center point (red marks). In SEM, it is difficult to do automatic detection of the alphanumeric character (indicated by a dotted black line, not the process of automatic detection). For this reason, the first character must be identified by the user and then given as input to the map. Each grid bar crossing surrounding the character is imaged (yellow remark at the bottom). Here, a different convolutional neural network is used to evaluate the probabilities of being a line on each crossing (c, second image, red marks). The identification of the center position of the crossing is very similar to the one in LM, here the intersections (c, third image, red dots) are identified after line detection, and the center point is stored as a position (c, third image, yellow dot). This process continues at each predicted landmark to give a list of landmarks (EM landmarks list). (d) A transformation is computed to register the positions from the LM and the EM landmarks lists (pink, black), which is then applied to the LM targets list (green) to predict the respective EM targets list (orange) across the sample at the FIB-SEM. (e) At the end of the experiment, the position of the cell can be validated using manual registration. FM (first image, top left) and SEM (second image, top right) images were superimposed manually using the cell contours. For this, the FM images were flipped, rotated, and scaled (first image, bottom left). The position of the LM target (white cross) is then compared with the predicted target in the SEM (black cross) (second image, bottom right). This overlay of SEM with LM images was repeated for each experiment, obtaining a final targeting accuracy of 5 ± 3 µm (RMSD over n = 10). Scale bars: (a) 200, 25, 25 µm; (b) 200, 100 µm with small window upper left corner 25, 50 µm; (c) all 100 µm; (e) all 50 µm.

Coordinate system mapping and automatic detection for the correlation strategy. (a) Cell of interest selected using fluorescence microscopy by scanning low magnification images (first and second image). In our experiments, we targeted the Golgi apparatus center of mass (a, third image, white cross). The image position is translated to stage position coordinates and stored in the “LM targets list” (green). (b) Simultaneously, reflected light images (b, first image) are stored, and later used to extract the stage coordinates of landmarks (LM landmarks list, pink). The image is analyzed and a line detector is applied (red lines). The intersection of the lines is used to find grid bar crossings (b, second image including inset). The corresponding detected edges are converted to lines that automatically mark 4 points (b, second image, red dots). Those points are used to determine the center point (second image, yellow dot), and they will be part of the “LM landmarks list.” By convention, the top left corner (yellow arrowhead) is named by associating its unique center point (yellow dot) with the alphanumeric identifier imprinted onto the glass dish bottom. To identify the alphanumeric character on the image, the reflected light image is automatically thresholded and cleaned (b, third right image) using a combination of traditional image analysis pipelines (see Fig. S1) and then passed through a convolutional neural network for classification, in this case, 8Q. (c) In the FIB-SEM, the strategy of mapping is repeated: scan images are taken by the Navigator module (c, first image), and the grid bar crossings are detected to calculate the center point (red marks). In SEM, it is difficult to do automatic detection of the alphanumeric character (indicated by a dotted black line, not the process of automatic detection). For this reason, the first character must be identified by the user and then given as input to the map. Each grid bar crossing surrounding the character is imaged (yellow remark at the bottom). Here, a different convolutional neural network is used to evaluate the probabilities of being a line on each crossing (c, second image, red marks). The identification of the center position of the crossing is very similar to the one in LM, here the intersections (c, third image, red dots) are identified after line detection, and the center point is stored as a position (c, third image, yellow dot). This process continues at each predicted landmark to give a list of landmarks (EM landmarks list). (d) A transformation is computed to register the positions from the LM and the EM landmarks lists (pink, black), which is then applied to the LM targets list (green) to predict the respective EM targets list (orange) across the sample at the FIB-SEM. (e) At the end of the experiment, the position of the cell can be validated using manual registration. FM (first image, top left) and SEM (second image, top right) images were superimposed manually using the cell contours. For this, the FM images were flipped, rotated, and scaled (first image, bottom left). The position of the LM target (white cross) is then compared with the predicted target in the SEM (black cross) (second image, bottom right). This overlay of SEM with LM images was repeated for each experiment, obtaining a final targeting accuracy of 5 ± 3 µm (RMSD over n = 10). Scale bars: (a) 200, 25, 25 µm; (b) 200, 100 µm with small window upper left corner 25, 50 µm; (c) all 100 µm; (e) all 50 µm.

This Feature Is Available To Subscribers Only

or Create an Account

Close Modal
Close Modal