Summer School on Imaging for Medical Applications
29 June - 4 July Sinaia Romania
Prof. Bart ter Haar Romeny
Eindhoven University of Technology, Netherlands
Northeastern University, Shenyang, China
Lecture: Brain-inspired computer vision
Duration: 4h
Our visual system is impressive in visual tasks. One quarter of the brain seems to be involved in its functioning.
Modern electrophysiological, optical and MRI techniques have given much insights in the details of its neural
circuits, connections and function. The first lecture will present a detailed overview of the first stages of the human
visual system, with focus on its geometrical task:
1. The visual system, its physiology, and a model as geometry inference engine
a.
b.
c.
d.
e.
Retina, LGN, cortex
Receptive fields, Gaussian derivative model
Axiomatic derivation of best aperture
Cortical columns, pinwheels
Eigenpatches
The receptive fields in the cortex can be modeled as multi-scale, regularized differential operators to high order. In
the second hour we discuss the construction of differential invariant features, like corners, ridges and T-junctions,
and how the multi-scale aspect can be exploited:
2. Multi-scale differential structure of images
a.
b.
c.
d.
Gauge coordinates
Differential invariants
Curvature, ridges, corners, T-junctions
Geometry-driven diffusion
The visual cortex turns out to be extremely well organized in a 2D array or so-called cortical columns, each with a
pinwheel structure, representing all orientations. They are connected over long distances, giving rise to the modeling
of contextual operations. In the third lecture we construct a mathematical model for this multi-orientation geometric
analysis:
3. Multi-orientation differential structure of images
a.
b.
c.
d.
Orientation scores
Cake kernels versus Gabor kernels
Enhancement and completion
Steerability of kernels
All these methods can be applied effectively to many areas of medical image analysis. We discuss a rich set of
applications for retinal image analysis. Diabetes is a major problem today, taking epidemic proportions. As vessels
start to leak, retinal images are a very cost-effective way to study the microvasculature in detail. The RetinaCheck
project is a large screening project, aiming to image 24 million people in Northeastern China for the early detection
of diabetic retinopathy. All methods discuss in the first lectures will be applied in the fourth and last lecture:
4. RetinaCheck: Quantitative analysis of retinal images for early diabetes detection
a.
b.
c.
Project description, partners
Vessel tracking, segmentation
Vessel curvature
d. Bifurcation detection
This course can only scratch the surface, but aims to give a concise introduction to the field, and inspire to further
reading and designing applications in brain-inspired ‘geometric reasoning’.
5. General conclusion of the course.
a. Questions and discussion
Reading material:
Lecture 1:
[1] David Hubel (1988): Eye, Brain & Vision. MIT Press.
URL: http://hubel.med.harvard.edu/index.html.
David Hubel, who received with Thorsten Wiesel the Nobel Prize for the pioneering work on discoveries in the
primary visual cortex, wrote this book for a broad audience. This book is highly recommended for first
reading. The book can be downloaded for free. The website also gives all papers by Hubel as free pdf.
[2] Helga Kolb, Ralph Nelson, Eduardo Fernandez, Bryan Jones (2014): Webvision.
URL: http://webvision.med.utah.edu/.
A detailed and well explained and illustrated account of the organization of the retina and the visual system.
Webvision summarizes recent advances in knowledge and understanding of the visual system through dedicated
chapters and evolving discussion to serve as a clearing house for all things related to retina and vision science.
Lecture 2:
[3] Bart terHaarRomeny (2004): Front-end vision and multi-scale image analysis. Springer.
URL: http://bmia.bmt.tue.nl/Education/Courses/FEV/book/index.html.
This interactive tutorial book is written as a series of Mathematica notebooks.
Lecture 3:
[4] BartterHaarRomeny (2011): Multi-scale and multi-orientation medical image analysis.
In: T.M. Deserno (Ed.), Biomedical Image Analysis, pp. 175-194. Berlin: Springer.
URL: http://bmia.bmt.tue.nl/Education/Courses/FEV/course/pdf/Romeny-MultiScale-MultiOrientation.pdf.
Lecture 4:
[5] Erik Bekkers, Remco Duits, Tos Berendschot and Bart terHaarRomeny (2014): A multi-orientation analysis
approach to retinal vessel tracking. Journal of Mathematical Imaging and Vision, 49(3), 583-610.
URL: http://bmia.bmt.tue.nl/Education/Courses/FEV/course/pdf/Bekkers-Tracking-JMIV2014.pdf.
[6] URL: www.retinacheck.org.