Medical Image Processing
Introduction
What Question Does It Ask?
Medical image processing enhances, segments, measures and compares X-ray, CT, MRI and ultrasound images to support diagnosis and treatment. It applies image processing and computer vision methods under the constraints of little data, expensive labels and patient safety.
At a glance
image processing plus clinical knowledge
CT, MRI, X-ray, ultrasound
segmentation, measurement, diagnostic suggestion
sensitivity, specificity, Dice
medium-high
little; labels are expensive
Introduction
A Short History
Röntgen discovers X-rays; the first medical image is an X-ray of his wife's hand.
Hounsfield's computed tomography (CT) scanner is introduced: cross-sections are obtained by computation.
Lauterbur publishes the first imaging paper using magnetic resonance, laying the foundation of MRI.
The DICOM 3.0 standard is published; devices from different vendors speak the same file format.
U-Net is published: an encoder-decoder network for segmentation with few labeled medical images.
The US FDA authorizes IDx-DR, which screens for diabetic retinopathy without a physician.
The Course
Core Concepts
X-ray, CT, MRI, ultrasound, PET: each a different physics.
A standard that carries the image and patient data together.
CT and MRI are 3D; voxels instead of pixels.
Preserving quality in low-dose CT and in MRI.
Outlining an organ, tumor or vessel.
Aligning two images (before/after, CT/MRI).
The computer flags suspicious regions for the physician.
Sensitivity, specificity, Dice; comparison with physicians.
The Course
A Typical Workflow
The Course
What Does the Course Cover?
- Imaging physics: X-ray, CT, MRI, ultrasound
- DICOM, NIfTI and medical image data
- Image reconstruction: filtered back-projection
- Denoising and image enhancement
- Classical segmentation: thresholds, regions, level sets
- Deep segmentation: U-Net and nnU-Net
- Rigid and deformable registration
- Atlas-based analysis and volumetry
- Classification and computer-aided diagnosis
- Learning from little data: augmentation, transfer
- Clinical validation: Dice, ROC, reader studies
- Regulation, privacy and clinical deployment
Skills you will gain
A typical semester; the order and coverage vary between universities.
Try It
Segment the Tumor in an MRI Slice
In a contrast-enhanced brain MRI slice the tumor appears bright; the radiologist wants its volume. Click inside the tumor to place a seed and change the tolerance: does the region cover the tumor or leak into healthy tissue?
Context
Prerequisites
Courses worth knowing before this one, and why they are needed.
| Course | Why it is needed |
|---|---|
| Image Processing | denoising, segmentation, registration |
| Computer Vision | lesion detection, classification |
| Programming | working with multidimensional arrays in Python |
| Physics | basics of X-ray absorption and magnetic resonance |
| Intro to anatomy and physiology | recognizing organs and how they appear in images |
Context
Are You Ready for This Course?
Ask yourself; for every "no", the right-hand column shows where to start.
| Ask yourself | If not, start with |
|---|---|
| Can you apply filtering, thresholding and edge detection to an image? | Image Processing |
| Can you explain how a convolutional neural network is trained? | Deep Learning |
| Can you compute and interpret sensitivity and specificity? | Probability and Statistics |
| Can you slice a three-dimensional NumPy array in Python? | An intro programming course, then a short NumPy tutorial |
Context
Neighbouring Courses
Overlap
Overlapping Courses
| Relation | Course | Shared ground |
|---|---|---|
| application | This course applies the methods of Image Processing | denoising, segmentation, registration |
| application | This course applies the methods of Computer Vision | lesion detection, classification |
| shares methods | Shares methods with Explainable AI and AI Ethics | explainability and accountability in diagnosis |
Differences
Where It Differs (1/2)
| Criterion | Medical Imaging | Image Processing | Computer Vision |
|---|---|---|---|
| Core question | How do X-ray, CT and MRI images help diagnosis and treatment? | How is an image enhanced, transformed and divided into regions? | How does a computer understand what is in an image or a video? |
| Approach | image processing plus clinical knowledge | pixel-level transformations | extracting meaning from images |
| Input | CT, MRI, X-ray, ultrasound | image | image or video |
| Output | segmentation, measurement, diagnostic suggestion | processed image or regions | label, box, depth, 3D structure |
| Success measure | sensitivity, specificity, Dice | visual quality, PSNR | accuracy, mAP, IoU |
| Math load | medium-high | medium: convolution, Fourier | high: geometry, linear algebra |
| Data needs | little; labels are expensive | little; learning not required | many labeled images |
- Image Processing: Image processing provides the general methods; this course applies them to 3D medical volumes, with device physics and clinical constraints.
- Computer Vision: Vision works with millions of labeled photos; in medical imaging labels are scarce, need experts, and errors are costly.
Differences
Where It Differs (2/2)
| Criterion | Medical Imaging | Explainable AI & Ethics | Bioinformatics |
|---|---|---|---|
| Core question | How do X-ray, CT and MRI images help diagnosis and treatment? | When can we trust an AI system's decision, and who is responsible for the outcome? | How are biological sequences and molecular data analyzed computationally? |
| Approach | image processing plus clinical knowledge | human-centered evaluation | biology, algorithms and statistics |
| Input | CT, MRI, X-ray, ultrasound | model, its data and context of use | DNA, protein sequences, expression data |
| Output | segmentation, measurement, diagnostic suggestion | explanation, audit report, principles | alignment, structure, function prediction |
| Success measure | sensitivity, specificity, Dice | interpretability, fairness, accountability | biological validity |
| Math load | medium-high | low-medium | medium-high |
| Data needs | little; labels are expensive | the model's own data | huge sequence datasets |
- Explainable AI & Ethics: The ethics course treats explainability in general; this course uses it for physician trust and accountability.
- Bioinformatics: Bioinformatics looks at sequence and molecular data; medical image processing looks at images of organs and tissue.
Practice
Applications
Finding lung nodules and fractures.
Tumor segmentation, radiotherapy planning.
Screening retinal images for retinopathy.
Lesion and volume measurement in brain MRI.
Counting cells in tissue sections.
Preoperative 3D models and navigation.
Practice
Risks and Ethics
A model can learn scanner labels or hospital markers instead of disease; this has been shown in chest X-ray models.
Performance drops with other scanners, protocols and patients; validation at a single site is not enough.
DICOM headers hold identifiers and faces can be rebuilt from head MRI; anonymization is more than deleting a name.
Clinicians may miss a lesion the model did not flag; a support tool can loosen human scrutiny.
In depth: Explainable AI and AI Ethics.
Practice
Tools and Datasets
Tools and libraries
Well-known datasets
Practice
Common Misconceptions
| Common belief | In fact |
|---|---|
| AI will soon replace radiologists. | Systems support narrow tasks; diagnosis, context and responsibility stay with the physician. |
| High accuracy means clinical success. | For rare diseases sensitivity and specificity matter; performance may drop at another hospital. |
| A medical image is an ordinary photo. | CT and MRI are often 3D, and pixel values carry physical meaning (Hounsfield units). |
| Collecting lots of data is easy. | Labels take expert time, and the data is protected by privacy law. |
Next Steps
After This Course
Courses that build on or extend this one
Related careers
Next Steps
Further Reading
- Suetens, Fundamentals of Medical Imaging, 3rd ed., Cambridge University Press, 2017.
- Bankman (ed.), Handbook of Medical Image Processing and Analysis, 2nd ed., Academic Press, 2008.
- Ronneberger, Fischer and Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation", MICCAI, 2015.
- Litjens et al., "A Survey on Deep Learning in Medical Image Analysis", Medical Image Analysis 42, 2017.