Image Processing

Guide to AI and Data Courses · Part III: Signals, Images and Audio
What is an image?EnhancementFilters and convolutionEdges and segmentationCompressionRelated courses
Sercan KÜLCÜ · Giresun University · Computer Engineering

Introduction

What Question Does It Ask?

How is an image enhanced, transformed and divided into regions?

Image processing studies methods that take an image and turn it into a new image or into regions: reducing noise, boosting contrast, finding edges and object boundaries. Both input and output are images; understanding what is in the image is the job of computer vision.

At a glance

Approach

pixel-level transformations

Input

image

Output

processed image or regions

Success measure

visual quality, PSNR

Math load

medium: convolution, Fourier

Data needs

little; learning not required

Introduction

A Short History

1920s

The Bartlane system sends digitally coded newspaper photos across the Atlantic by cable.

1957

Russell Kirsch makes the first digitally scanned photo: a 176×176-pixel image of his infant son.

1964

JPL corrects Ranger 7's lunar images by computer; digital image processing is born.

1979

Otsu publishes his method for picking a threshold from the histogram; still the most used thresholding.

1986

The Canny edge detector appears: noise-robust, thin and continuous edges.

1992

The JPEG standard is adopted; DCT-based lossy compression becomes the common language of photos.

The Course

Core Concepts

Pixels and channels

An image is a matrix of numbers; color has three channels.

Histogram

Brightness distribution; contrast via histogram equalization.

Spatial filter

Convolution with neighboring pixels: blurring, sharpening.

Frequency domain

2D Fourier transform; filtering periodic noise.

Edge detection

Brightness jumps via gradient, Sobel and Canny.

Thresholding

Separating foreground from background; Otsu's method.

Morphology

Erosion and dilation: cleaning up binary shapes.

Segmentation

Splitting an image into regions: region growing, watershed.

The Course

A Typical Workflow

Acquirecamera,scannerPreprocessnoise,contrastTransformfilter,FourierSegmentthreshold,edge, regionImageoutputclean imageor mask
Example: An old scanned document is first cleaned of specks with a median filter, its contrast is boosted by histogram equalization, it is binarized with Otsu's threshold, and the letters are repaired with morphological operations. The output is a clean image; reading the letters is the job of computer vision.

The Course

What Does the Course Cover?

  1. Image formation, sampling and quantization
  2. Point operations and histogram equalization
  3. Spatial filtering: smoothing and sharpening
  4. The 2D Fourier transform and frequency-domain filtering
  5. Image restoration: noise models, the Wiener filter
  6. Colour spaces and colour image processing
  7. Geometric transforms and interpolation
  8. Edge detection: gradients, Sobel, Canny
  9. Thresholding and Otsu's method
  10. Morphological operations
  11. Segmentation: region growing, watershed, labelling
  12. Image compression: the DCT and JPEG

Skills you will gain

building image pipelines with OpenCVsegmentation with scikit-imagedesigning filters and kernelsinterpreting histograms and spectrameasuring objects in ImageJ

A typical semester; the order and coverage vary between universities.

Try It

Finding Edges in a Noisy Image

A quality control camera returned a 32×32 grayscale image of a part; the sensor is noisy. Toggle the steps and slide the threshold: how many of the detected edges lie on a real shape boundary?

Each step is a real 3×3 convolution. Blur and lower the threshold a little: noise edges drop and F1 rises.
In image processing both input and output are images: filters transform pixels, they do not decide. The course teaches what each kernel does and how the order of steps changes the result.

Context

Prerequisites

Courses worth knowing before this one, and why they are needed.

CourseWhy it is needed
Digital Signal Processing2D signals, convolution, Fourier transform
Linear algebra and calculusmatrix operations, derivatives and gradients: the language of filters and edges
Programmingloops over 2D arrays in Python or C
Data Structuresqueues and union-find: region growing and connected-component labeling

Context

Are You Ready for This Course?

Ask yourself; for every "no", the right-hand column shows where to start.

Ask yourselfIf not, start with
Can you visit every element of a matrix with nested loops?An intro programming course, then a short NumPy tutorial
Can you explain what a partial derivative and a gradient are?Derivatives and partial derivatives from calculus
Can you multiply matrices and take a dot product?Linear Algebra
Can you explain what the frequency components of a signal are?Digital Signal Processing
If you can say "yes" to most of these, you are ready; one or two gaps can be closed during the semester.

Context

Neighbouring Courses

subfieldbuilds onshares methodsapplication

Overlap

Overlapping Courses

RelationCourseShared ground
builds onThis course builds on Digital Signal Processing2D signals, convolution, Fourier transform
builds onComputer Vision builds on this courseedges, filters, segmentation
applicationMedical Image Processing applies the methods of this coursedenoising, segmentation, registration
applicationRemote Sensing applies the methods of this coursemultispectral images, correction
shares methodsShares methods with Pattern Recognitionfeature extraction, template matching

Differences

Where It Differs (1/2)

CriterionImage ProcessingComputer VisionSignal Processing
Core questionHow is an image enhanced, transformed and divided into regions?How does a computer understand what is in an image or a video?How are sampled signals analyzed, filtered and transformed?
Approachpixel-level transformationsextracting meaning from imagesmathematical transforms and filters
Inputimageimage or videosampled signal
Outputprocessed image or regionslabel, box, depth, 3D structurefiltered signal, spectrum
Success measurevisual quality, PSNRaccuracy, mAP, IoUsignal-to-noise ratio, frequency response
Math loadmedium: convolution, Fourierhigh: geometry, linear algebrahigh: complex numbers, Fourier
Data needslittle; learning not requiredmany labeled imagesno learning; the signal suffices
  • Computer Vision: In image processing both input and output are images; in computer vision the input is an image and the output is meaning: a label, a box, depth.
  • Signal Processing: Signal processing works with a 1D time signal; image processing carries the same convolution and Fourier tools to two dimensions.

Differences

Where It Differs (2/2)

CriterionImage ProcessingComputer GraphicsMedical Imaging
Core questionHow is an image enhanced, transformed and divided into regions?How is a realistic image produced from the description of a scene?How do X-ray, CT and MRI images help diagnosis and treatment?
Approachpixel-level transformationsgeometry and light simulationimage processing plus clinical knowledge
Inputimagescene: models, lights, cameraCT, MRI, X-ray, ultrasound
Outputprocessed image or regionsimage or animationsegmentation, measurement, diagnostic suggestion
Success measurevisual quality, PSNRrealism, frame ratesensitivity, specificity, Dice
Math loadmedium: convolution, Fourierhigh: linear algebra, geometrymedium-high
Data needslittle; learning not requirednonelittle; labels are expensive
  • Computer Graphics: Graphics produces an image from a scene description; image processing takes an existing image and transforms it.
  • Medical Imaging: Medical image processing applies this course's methods to medical images such as CT and MRI, under clinical constraints.

Practice

Applications

Photo editing

Brightness, sharpness, noise reduction.

Document scanning

Deskewing, converting to black and white.

Compression

JPEG, PNG and video coding.

Industrial inspection

Isolating scratches and stains on products.

Microscopy

Segmenting and counting cells.

Preprocessing

Clean input for vision and medical models.

Practice

Risks and Ethics

Image forgery

The same tools can remove or add objects in a photo, and an edited image can be passed off as evidence.

Corrupted evidence

Lossy JPEG and sharpening change detail in forensic or medical images; the original file must be kept.

Invented detail

Super-resolution and denoising can produce detail that was never in the scene, and it may be taken as real.

Hand-tuned thresholds

Thresholds and parameters picked by looking at one image fail silently when the lighting changes.

In depth: Explainable AI and AI Ethics.

Practice

Tools and Datasets

Tools and libraries

OpenCVscikit-imagePillowImageJ / FijiMATLAB Image Processing ToolboxGIMP

Classic test images and datasets

USC-SIPI test imagesKodak image setBSDS500 (Berkeley segmentation)DIV2K (super-resolution)Cameraman and Peppers
In image processing a single image is often enough: the method is defined in advance, not learned from data. Large labeled datasets are what computer vision needs.

Practice

Common Misconceptions

Common beliefIn fact
Image processing and computer vision are the same thing.Image processing transforms the image; vision extracts meaning from it. One is a step before the other.
Sharpening brings back lost detail.Sharpening accentuates existing edges; detail that was never recorded does not come back.
Deep learning does everything now.Classical methods are still essential in calibration, compression, preprocessing and fast industrial inspection.
Enlarging an image increases its resolution.Interpolation creates new pixels but no new information; the image just gets bigger.

Next Steps

After This Course

Courses that build on or extend this one

Related careers

Image processing engineerEmbedded vision engineerQuality control systems engineerBiomedical imaging specialist
Image processing is the common first step for every course that works with images. To extract meaning from images, move on to computer vision; for applications, to medical image processing or remote sensing. In the opposite direction, computer graphics studies how to produce images.

Next Steps

Further Reading

  • Gonzalez and Woods, Digital Image Processing, 4th ed., Pearson, 2018.
  • Jain, Fundamentals of Digital Image Processing, Prentice Hall, 1989.
  • Burger and Burge, Digital Image Processing: An Algorithmic Introduction, 3rd ed., Springer, 2022.
  • Canny, "A Computational Approach to Edge Detection", IEEE Transactions on Pattern Analysis and Machine Intelligence 8(6), 1986.

The End

Chapter 17: Image Processing
← Previous chapter · Map of the field · Next chapter →
1 / 1 Sercan KÜLCÜ. All rights reserved.