Skip to content

Superresolution Microscopy Research Practicum

  • authors
    • Bernd Rieger, b.rieger@tudelft.nl, room: F266
    • Ronald Ligteringen, r.ligteringen@tudelft.nl, room: F261
  • date: April 3, 2020

Introduction

As a result of the COVID-19 virus in March 2020 this Research Practicum (RP) has been converted from an onsite lab to a home lab. This means that all work can and should be done from home with your laptop or desktop and a smartphone-camera or movie-camera. The camera is used to record images shown on your computer. The recorded video must be uploaded to your computer where you will write a Python program to process the video. This program together with the results and a report are the final deliveries for this RP.

This manual will help you install and use the python programming environment and the python script used in this practicum to display the images. Furthermore a few pointers are given on how to use the DIPlib image processing package for python.

In the Python manual a very short introduction is given to different aspects of the python environment used in this practicum. Note this manual does not provide a course about python!

Packages typically used in this practical are PyDIP, numpy, matplotlib, ipywidgets and cv2 (a.k.a. opencv). To learn about these packages look at the websites given below and check their usage in the show_movies.py script provided in this practicum.

The PyDIP package will be used to do all the image processing in this practical. It is part of DIPlib library for quantitative image analysis developed in our research group Quantitative Imaging. In chapter PyDIP package some of the functions available in PyDIP and necessary for this practical are described.

PyDIP package

The purpose of this practical is to write a program to find locations of the different blobs you see in each time frame. These frames can be shown and recorded with show_movies and capture_movies. The file-format of the recorded image timeseries is ICS. This is the preferred format used in DIPlib: https://svi-opensource.github.io/libics/.

Below some recommendations are provided that can help you solve this task:

  • use the DIPlib library with the python package PyDIP. Follow the installation instructions in the DIPlib manual to install this package on your computer

Note

These instructions are currently only provided for Windows.

import PyDIP as dip
  • almost all functions available in DIPlib can be used via PyDIP. A description of these functions can be found here: https://diplib.github.io/diplib-docs/. You can also find a (sometimes very brief) description via the help function in python:
help(dip)
  • bring in the recorded images with dip.ImageReadICS():
imgs = dip.ImageReadICS('C:/rp/diplib/images/chromo3d.ics')

Note

IPython (Jupyter) allows for autocompletion with the [TAB] key! Also note the use of forward-slashes / in the filename.

  • typically noise suppression can be done with the function dip.Gauss(). Use smoothing parameters like [2, 2, 0]. Note the zero as you do not want to smooth along the time axis.
  • use a for-loop to iterate over all time frames of the recorded image timeseries
  • the dip.IsodataThreshold() and dip.Label() functions can be used to segment the blobs from the background:
  • to find the center of mass of the blobs you can use the function dip.MeasurementTool.Measure() together with a mask image (from the threshold) and the original intensity image. This very important Measure function also allows you to find the size of objects - make use of that
  • you can store the found coordinates into an array for each time frame and add additional coordinates in the loop
  • after you have localized all blobs and stored them in an array you can use the following snippet to generate an image from these coordinates:
reconstruct = dip.Image(sizes=[640, 480], dt='SFLOAT')
reconstruct.Fill(0)
for ii in coordinates:
    reconstruct[round(ii[0]), round(ii[1])] += 1

Some useful functions:
Sizes(), Squeeze(), Fill(), Sum(), Overlay(), ContrastStretch(), Show()

Setting up the practicum

For this practicum you will display a sequence of images on your computerscreen and record those images with your camera. This recorded video will then be processed with your Python script using PyDIP functions.

To display the image sequence you need to download the following:

  1. download all images by clicking on this link: ftp://qiftp.tudelft.nl/rp/Movies.zip

    Warning

    This is a large file (8.4GB), make sure you have enough space on your computers disk and enough time to wait for the download to complete!

    Tip

    You can also use FileZilla, fill in the hostname qiftp.tudelft.nl and click Quickconnect. In the right upper window click the rp directory, right-click in the right lower window on the file Movies.zip and select Download.

  2. unzip Movies.zip and make note of the location of the resulting Movies directory (the full path)

  3. download the Python scripts needed to display the image sequence by clicking on this link: https://gitlab.tudelft.nl/rligteringen/rp_sm/-/archive/master/rp_sm-master.zip
  4. unzip rp_sm-master.zip in your working directory (i.e. the directory where you will be developing your scripts for this practicum)

Warning

The following instructions have not yet been rewritten for working from home!

  1. after logging in with your netid go the the directory c:\Users\<netid> and create the directory rp. In this location you will store your programs and data
  2. copy all ipynb- and py-files from C:\rp\ to c:\Users\<netid>\rp\. These scripts can be used to show all images and record videos with the webcam. Do not copy the diplib and Movies directories!
  3. start Jupyter via Start Menu -> Anaconda3 -> Jupyter Notebook
  4. after the notebook-webpage has opened navigate to your rp directory in the filemanager
  5. execute both the show_movies.ipynb notebook and the capture_movies.ipynb notebook to create your datafiles (movies)
    • check the code in both notebooks and adjust to your needs
    • check the key-commands for recording movies
    • After pressing the ‘s’-key (save) in capture_movies.ipynb wait for the ‘Saving…’ message at the bottom of the window to disappear before closing the window otherwise the saved file will be corrupt! Note that during the save-operation the window title might show ‘(Not responding)’. Please disregard this message and wait for the save-operation to finish.
    • note that both notebooks actually call python scripts. These scripts can also be started directly from the command line but need some small modifications to allow for parameters. To execute these scripts start Start Menu -> Anaconda3 -> Anaconda Prompt and for instance enter python show_movies_anim.py
  6. create a new notebook to write your own program or use the Spyder IDE via Start Menu -> Anaconda3 -> Spyder

Finally, this is an open Research Practicum in the sense that you can decide for yourself on how to solve the problem. You do not need to use the given scripts. They can also be viewed as examples on how to develop your own solution!

Using the DIPlib viewer

There are two ways to show images in PyDIP. The easiest way is to call the Show() method for any dipimage-object

imgs.Show()

png

Note: the chromo3d.ics file contains 16 frames and the Show() method squeezes and stretches these frames

It is also possible to start the DIPlib viewer. For this you can use the following snippet

dip.viewer.Show(imgs)
dip.viewer.Spin()

Note: always call the Spin() function after Show() and always close the viewer window with [CTRL]-w

Make sure you learn the different aspects of this viewer via this website: https://diplib.github.io/diplib-docs/group__viewer.html