Session 4 - Cell Detection

Goal: Learn 2 methods for detecting cells in multiplex images, including the pros and cons of each

Sharing and downloading QuPath projects

If you ever fall behind during the training course, you can download a pre-cooked project from Google Drive. (named by session name- right now, you would want “Proj1 - Session 4”, which will have the tissue and tumor annotations). Alternatively, you may one day need to share a QuPath project with a collaborator or between computers or store it on a backup drive for safekeeping.

If you want to continue using the tissue segmentation you just created- great! This is just here as a reference in case you need it some other time.

Transferring a QuPath project

To transfer a QuPath project, you must share: - The entire project folder that you created when you first began working, including the .qpproj file and all subfolders - Separately, the raw images files The benefit of keeping the images files separate from the QuPath project folder is that you can trasnfer the large image files once, and then share versions of the much smaller project folder back and forth as often as you’d like.

Here are the contents of the “Proj1 - Session 4” folder:

Opening a shared project

1. To open the project, drag either the entire project folder or the .qpproj file into the main QuPath window.

   • The project does not store the images themselves, only a link to where it believes the image is located.

   • When you open the downloaded project, it will not see the image files in the same place. You’ll get a message with some scary-looking red text that says the files are missing.

2. Click Search

3. Navigate to wherever the image files are stored on your computer. You just need to find the outer folder- the window will not show you the file contents.

4. Click Select Folder . QuPath will search through the folder and all subfolders to find image files with a matching name.

5. Click Apply changes

Cell Method 1- QuPath’s Built-In Cell Detection

1. Select the whole tissue annotation

2. Analyze > Cell detection > Cell detection
   

Start with these settings:

• Oh No! My Tumor Annotation Disappeared! When you detect cells, it deletes the ‘child’ objects in the hierarchy. Don’t worry, we will get it back!

3. Spend a few minutes adjusting the settings and rerunning the cell detection. Some settings will look better for the tumor cells, some will look better for the inflammatory cells. Find what looks best overall.
   If you are interested in more detail about the settings themselves, you can check out the FS2K2023 session on cell detection, or read about them here.

4. Click on the tissue annotation in the Annotation tab. In the measurements table, it tells you the total number of cells - as Num detections. The area is in square microns. To convert to square mm, multiply by \(1000*1000 = 1e6\). You can calculate cell density as: \(NumCells / Area * 1000000 = cells / mm^2\)

5. You can visualize the nucleus, the cell membrane, or both by right click > Cells and choosing one of the options there. You can turn the detection outlines on and off with D and fill them in the F (or these buttons):
   

Understanding Detection Measurements

1. Select any cell (usually double click, though this can vary) and look at the measurements table in the Annotations tab. It gives you a variety of shape and intensity measurements for each cell.

2. Open up the detections measurements table Measure > Show detection measurements. Things you can do with this table:

• Sort by any measurement by clicking on the column header

• Double click on a cell to find it in the image

• Double click on a cell in the image to jump to that cell in the measurement table

• Create a histogram of a measurement

• Filter by class (will get to this later)

Method 2- InstanSeg

To use InstanSeg, you must have QuPath verison 0.6.0. The current most up to date is 0.6.0-rc3.
See the debut of InstanSeg at FS2K2023!

One-time setup for InstanSeg

1. Extensions > Deep Java Library > Manage DJL Extensions

2. Under PyTorch (default) click Check / download

This may take a while to download, the orange circle will turn green when it is done. You can then close this window.

NOTE: We are not covering using the GPU here. For further information about PyTorch GPU acceleration, see the documentation

3. Start up Instanseg with: Extensions > InstanSeg > Run instanseg
   

4. The first time you run InstanSeg, it will ask you about checking the internet for new models. Click “Allow once”.
   

5. InstanSeg will need to download some pre-trained models.

6. Choose the location on your computer where you want them saved by clicking on the folder icon

7. Select the fluorescence_nuclei_and_cells model from the dropdown

8. Click the download button.

9. If the model downloads correctly, the cloud icon on the left side of the dropdown will disappear.

   
   Note: your buttons will look slightly different because I had previously downloaded the models

Running InstanSeg

1. We’re going to save the built-in cell detection results so that we can compare them to the InstanSeg results. The easiest way to do this is to duplicate the image.

   1. Save your work! File > Save or Ctrl + S

   2. In the Projects tab, right click on the current image > Duplicate image. Name the new image ‘LungImg2.ome.tiff_Builtin’ or similar. Keep the “Also duplicate data files” option checked.

   3. A new image entry will be created in the Project tab.
      

2. Delete the existing cells with Objects > Delete > Delete all detections

3. Start up Instanseg with: Extensions > InstanSeg > Run instanseg

4. Click on the dropdown “Input channels”. Select only those you believe will be helpful for segmenting cells. You should always include the nuclear marker (Hoechst), and whatever membrane and cytoplasmic markers that show cell boundaries. Definitely uncheck the two blank channels.

5. UNCHECK Make measurements

6. Draw a SMALL annotation (~100 cells) around a region that contains tumor and stroma. With that annotation selected click Run

7. If you are unhappy with the results, try adjusting the input channels. Then, draw rectangles in different regions of the image to test the settings. Iterate until you are satisfied or until the results stop improving. It’s generally not worth tweaking the other input parameters unless you know a lot about your computer hardware.

8. When you have your final settings, delete all rectangles that you made for testing. Select the tissue annotation and run InstanSeg one more time (this will be slow)
   

Comparing Results of different segmentation methods

1. Save your work!

2. Use Multiview Mode. On the image, Right Click > Multi-view > Set grid size > Grid 1 x 2 (horizontal)

3. A new viewer will be created to the right. Click on it (it will be subtly outlined in red). Double click on the image LungImg2.ome.tiff_Builtin to open it in the new viewer.

4. Synchronize the viewers

   1. Right click > Multi-view > Uncheck 'Synchronize Viewers' (unintuitive but important for resetting them)

   2. Click on one of the images. Press the full screen button. . Repeat for the other side

   3. Right click > Multi-view > Check 'Synchronize Viewers'

   4. Open Brightness / Contrast

   5. Check “Apply to similar images” at the bottom
      Full method:

   
   

5. Zoom in to any spot. Compare the results of the built-in (Watershed-based) cell detection and that of InstanSeg. Make sure you look at different cells types and many regions. Only you can determine if the extra time needed for InstanSeg is worth the increased accuracy for your project!
   

6. To close the multiviewer, right click on the image on the right > Multi-view > Close Viewer. Then, right click on the left image > Multi-view > Set grid size > Grid 1x1 (single viewer)