Personnel

Instructor:	Selim Aksoy (Office: EA 422, Email: )
TA:	Hüseyin Gökhan Akçay (Office: EA 427, Email: akcay[at]cs.bilkent.edu.tr)

Course Information

Schedule:	Tue 10:40-12:30, Fri 8:40-10:30 (EB 201)
Office hours:	Selim Aksoy: Tue 16:40-17:30, Fri 15:40-16:30 Hüseyin Gökhan Akçay: TBD
Catalog description:	Image acquisition, sampling and quantization. Spatial domain processing. Image enhancement. Texture analysis. Edge detection. Frequency domain processing. Color image processing. Mathematical morphology. Image segmentation and region representations. Statistical and structural scene descriptions. Applications.
Prerequisites:	Good background on high-level programming, data structures, linear algebra, and vector calculus. No prior knowledge of image processing or computer vision is assumed.
Syllabus:	Make sure you read the syllabus for course details.

Texts

L. G. Shapiro and G. C. Stockman, Computer Vision, Prentice Hall, 2001.
R. Szeliski, Computer Vision: Algorithms and Applications, Springer 2010. (local copy)
R. C. Gonzalez and R. E. Woods, Digital Image Processing, 3rd edition, Prentice Hall, 2008.
D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Prentice Hall, 2002.
D. H. Ballard and C. M. Brown, Computer Vision, Prentice Hall, 1982.

Lectures

Topics	Contents
Introduction [ Slides ]	Topics: Overview Example applications
Digital Image Fundamentals [ Slides ]	Topics: Acquisition, sampling, quantization Image enhancement Image formats Linear algebra and MATLAB review Readings: SS Ch 1, 2 GW Ch 1, 2, 3.1-3.4 References: R. C. Gonzales, R. E. Woods, "Review material and slides on linear algebra, probability, and linear systems," 2002. Tour of the Electromagnetic Spectrum Book at NASA (local copy) Software: Matlab code examples for processing images Matlab code examples for histogram enhancement
Binary Image Analysis [ Slides: Part 1 \| Part 2 ]	Topics: Pixels and neighborhoods Mathematical morphology Connected components analysis Automatic thresholding Readings: SS Ch 3.1-3.5, 3.8 GW Ch 2.5, 9.1-9.5, 10.3 References: M. Sezgin, B. Sankur, "Survey Over Image Thresholding Techniques and Quantitative Performance Evaluation" (local copy), Journal of Electronic Imaging, 13(1):146-165, January 2004. Software: Matlab code examples for binary image analysis Mathematical morphology demos
Linear Filtering [ Slides: Part 1 \| Part 2 ]	Topics: Spatial domain filtering Frequency domain filtering Image enhancement Readings: SS Ch 5.1-5.5, 5.10-5.11 GW Ch 3.5-3.8, 4 Software: Matlab code examples for filtering
Edge Detection [ Slides ]	Topics: Edges, lines, arcs Hough transform Readings: SS Ch 5.6-5.8, 10.3-10.4 GW Ch 10.1-10.2 References: A. Fitzgibbon, M. Pilu, R. B. Fisher, "Direct Least Square Fitting of Ellipses" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 21(5):476-480, May 1999. Y. Li, L. G. Shapiro, "Object Recognition for Content-Based Image Retrieval" (local copy), Lecture Notes in Computer Science, 2004. S. Kiranyaz, M. Ferreira, M. Gabbouj, "Automatic Object Extraction Over Multiscale Edge Field for Multimedia Retrieval" (local copy), IEEE Trans. on Image Processing, 15(12):3759-3772, December 2006. M. S. Costa, L. G. Shapiro, "3D Object Recognition and Pose with Relational Indexing" (local copy), Computer Vision and Image Understanding, 79(3):364-407, September 2000. Software: Edge detection demo by Yi Li Line and arc finding demo by Yi Li Building recognition demo by Yi Li
Local Feature Detectors [ Slides ]	Topics: Corners and other interest points Invariants References: C. Schmid, R. Mohr, C. Bauckhage, "Evaluation of Interest Point Detectors" (local copy), International Journal of Computer Vision, 37(2):151-172, June 2000. K. Mikolajczyk, C. Schmid, "Scale & Affine Invariant Interest Point Detectors" (local copy), International Journal of Computer Vision, 60(1):63-86, October 2004. D. G. Lowe, "Distinctive Image Features From Scale-Invariant Keypoints" (local copy), International Journal of Computer Vision, 60(2):91-110, November 2004. K. Mikolajczyk, C. Schmid, "A Performance Evaluation of Local Descriptors" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(10):1615-1630, October 2005. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, L. Van Gool, "A Comparison of Affine Region Detectors" (local copy), International Journal of Computer Vision, 65(1-2):43-72, November 2005. Local invariant features tutorial by Schmid and Lowe at CVPR 2003 N. Snavely, S. M. Seitz, R. Szeliski, "Photo tourism: Exploring photo collections in 3D" (local copy, web page), ACM Transactions on Graphics (SIGGRAPH Proceedings), 25(3):835-846, 2006. I. Simon, N. Snavely, S. M. Seitz, "Scene Summarization for Online Image Collections" (local copy, web page), IEEE International Conference on Computer Vision, October 14-20, 2007. Software: Affine covariant features by University of Oxford SIFT keypoint detector by David Lowe Affine interest point detectors by INRIA Rhone-Alpes Affine invariant region detector by Timor Kadir VLFeat open source library Aperture problem demonstration
Color Image Processing [ Slides ]	Topics: Color spaces and conversions Readings: SS Ch 6.1-6.5 GW Ch 6
Texture Analysis [ Slides ]	Topics: Statistical approaches Structural approaches Readings: SS Ch 7 GW Sec 11.3.3
Image Segmentation [ Slides ]	Topics: Histogram-based approaches Clustering-based approaches Region growing Split-and-merge Morphological approaches Graph-based approaches Readings: SS Ch 10.1 GW Ch 10.4-10.5 References: J. Shi, J. Malik, "Normalized Cuts and Image Segmentation" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(8):888-905, August 2000. C. Carson, S. Belongie, H. Greenspan, J. Malik, "Blobworld: Image Segmentation Using Expectation-Maximization and Its Applications to Image Querying" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(8):1026-1038, August 2002. Y. Deng, B. S. Manjunath, "Unsupervised Segmentation of Color-Texture Regions in Images and Video" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(8):800-810, August 2001. P. Soundararajan, S. Sarkar, "An In-Depth Study of Graph Partitioning Measures for Perceptual Organization" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(6):642-660, June 2003. Image segmentation research at UC Santa Barbara Graph-based image segmentation tutorial by Shi, Fowlkes, Martin, Sharon at CVPR 2004 L. Vincent, P. Soille, "Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(6):583-598, June 1991. C. Vachier, F. Meyer, "The Viscous Watershed Transform" (local copy), Journal of Mathematical Imaging and Vision, 22(2-3):251-267, 2005. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, "SLIC Superpixels Compared to State-of-the-art Superpixel Methods" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 34(11):2274-2282, May 2012. Software: Berkeley segmentation engine Normalized cuts segmentation by Jianbo Shi JSEG: segmentation of color-texture regions by UC Santa Barbara Edgeflow segmentation by UC Santa Barbara
Representation and Description [ Slides ]	Topics: Image representations and descriptors Region representations and descriptors Readings: SS Ch 10.2, 3.7 GW Ch 11 References: D. Zhang, G. Lu, "Review of Shape Representation and Description Techniques" (local copy), Pattern Recognition, 37(1):1-19, January 2004.
Pattern Recognition Overview [ Slides: Part 1 \| Part 2 ]	Topics: Brief introduction to pattern recognition Readings: SS Ch 4 GW Ch 12.1-12.2 References: A. K. Jain, R. P. W. Duin, J. Mao, "Statistical Pattern Recognition: A Review" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(1):4-37, January 2000. Software: K-means clustering demo by Yi Li
Case Studies [ Slides: Part 1 \| Part 2 ]	Topics: Image retrieval Image classification Object recognition References: Y. Liu, D. Zhang, G. Lu, W.-Y. Ma, "A Survey of Content-Based Image Retrieval with High-Level Semantics" (local copy), Pattern Recognition, 40(1):262-282, January 2007. X. S. Zhou, T. S. Huang, "Relevance Feedback in Image Retrieval: A Comprehensive Review" (local copy), Multimedia Systems, 8(6):536-544, April 2003. S. Antani, R. Kasturi, R. Jain, "A Survey on the Use of Pattern Recognition Methods for Abstraction, Indexing and Retrieval of Images and Video" (local copy), Pattern Recognition, 35(4):945-965, April 2002. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain, "Content-Based Image Retrieval at the End of the Early Years" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1349-1380, December 2000. Y. Rui, T. S. Huang, S.-F. Chang, "Image Retrieval: Current Techniques, Promising Directions, and Open Issues" (local copy), Journal of Visual Communication and Image Representation, 10(1):39-62, March 1999. Y. Rubner, C. Tomasi, L. J. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval" (local copy), International Journal of Computer Vision, 40(2):99-121, November 2000. A. Vailaya, M. A. T. Figueiredo, A. K. Jain, H.-J. Zhang, "Image Classification for Content-Based Indexing" (local copy), IEEE Trans. on Image Processing, 10(1):117-130, January 2001. L. Fei-Fei, P. Perona, "A Bayesian Hierarchical Model for Learning Natural Scene Categories" (local copy), IEEE Conference on Computer Vision and Pattern Recognition, 2:524-531, June 20-25, 2005. P. Quelhas, F. Monay, J.-M. Odobez, D. Gatica-Perez, T. Tuytelaars, "A Thousand Words in a Scene" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 29(9):1575-1589, September 2007. D. Gokalp, S. Aksoy, "Scene Classification Using Bag-of-Regions Representations", IEEE Conference on Computer Vision and Pattern Recognition, Beyond Patches Workshop, June 23, 2007. M. R. Boutell, J. Luo, C. M. Brown, "Factor Graphs for Region-Based Whole-Scene Classification" (local copy), IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Semantic Learning Applications in Multimedia, June 17, 2006. Y. Li, J. A. Bilmes, L. G. Shapiro, "Object Class Recognition using Images of Abstract Regions" (local copy), IAPR International Conference on Pattern Recognition, 1:40-43, August 23-26, 2004. Y. Li, L. G. Shapiro, J. A. Bilmes, "A Generative/Discriminative Learning Algorithm for Image Classification" (local copy), IEEE International Conference on Computer Vision, 2:1605-1612, October 17-21, 2005. Recognizing and learning object categories tutorial by Fei-Fei, Fergus and Torralba at ICCV 2009 Visual object recognition tutorial by Leibe and Grauman at AAAI 2008 (local copy)

Topics

Introduction

[ Slides ]

Topics:

Overview
Example applications

Digital Image Fundamentals

[ Slides ]

Topics:

Acquisition, sampling, quantization
Image enhancement
Image formats
Linear algebra and MATLAB review

Readings:

SS Ch 1, 2
GW Ch 1, 2, 3.1-3.4

References:

R. C. Gonzales, R. E. Woods, "Review material and slides on linear algebra, probability, and linear systems," 2002.
Tour of the Electromagnetic Spectrum Book at NASA (local copy)

Software:

Binary Image Analysis

[ Slides: Part 1 | Part 2 ]

Topics:

Pixels and neighborhoods
Mathematical morphology
Connected components analysis
Automatic thresholding

Readings:

SS Ch 3.1-3.5, 3.8
GW Ch 2.5, 9.1-9.5, 10.3

References:

M. Sezgin, B. Sankur, "Survey Over Image Thresholding Techniques and Quantitative Performance Evaluation" (local copy), Journal of Electronic Imaging, 13(1):146-165, January 2004.

Software:

Linear Filtering

[ Slides: Part 1 | Part 2 ]

Topics:

Spatial domain filtering
Frequency domain filtering
Image enhancement

Readings:

SS Ch 5.1-5.5, 5.10-5.11
GW Ch 3.5-3.8, 4

Software:

Matlab code examples for filtering

Edge Detection

[ Slides ]

Topics:

Edges, lines, arcs
Hough transform

Readings:

SS Ch 5.6-5.8, 10.3-10.4
GW Ch 10.1-10.2

References:

A. Fitzgibbon, M. Pilu, R. B. Fisher, "Direct Least Square Fitting of Ellipses" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 21(5):476-480, May 1999.
Y. Li, L. G. Shapiro, "Object Recognition for Content-Based Image Retrieval" (local copy), Lecture Notes in Computer Science, 2004.
S. Kiranyaz, M. Ferreira, M. Gabbouj, "Automatic Object Extraction Over Multiscale Edge Field for Multimedia Retrieval" (local copy), IEEE Trans. on Image Processing, 15(12):3759-3772, December 2006.
M. S. Costa, L. G. Shapiro, "3D Object Recognition and Pose with Relational Indexing" (local copy), Computer Vision and Image Understanding, 79(3):364-407, September 2000.

Software:

Local Feature Detectors

[ Slides ]

Topics:

Corners and other interest points
Invariants

References:

C. Schmid, R. Mohr, C. Bauckhage, "Evaluation of Interest Point Detectors" (local copy), International Journal of Computer Vision, 37(2):151-172, June 2000.
K. Mikolajczyk, C. Schmid, "Scale & Affine Invariant Interest Point Detectors" (local copy), International Journal of Computer Vision, 60(1):63-86, October 2004.
D. G. Lowe, "Distinctive Image Features From Scale-Invariant Keypoints" (local copy), International Journal of Computer Vision, 60(2):91-110, November 2004.
K. Mikolajczyk, C. Schmid, "A Performance Evaluation of Local Descriptors" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 27(10):1615-1630, October 2005.
K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, L. Van Gool, "A Comparison of Affine Region Detectors" (local copy), International Journal of Computer Vision, 65(1-2):43-72, November 2005.
Local invariant features tutorial by Schmid and Lowe at CVPR 2003
N. Snavely, S. M. Seitz, R. Szeliski, "Photo tourism: Exploring photo collections in 3D" (local copy, web page), ACM Transactions on Graphics (SIGGRAPH Proceedings), 25(3):835-846, 2006.
I. Simon, N. Snavely, S. M. Seitz, "Scene Summarization for Online Image Collections" (local copy, web page), IEEE International Conference on Computer Vision, October 14-20, 2007.

Software:

Color Image Processing

[ Slides ]

Topics:

Color spaces and conversions

Readings:

SS Ch 6.1-6.5
GW Ch 6

Texture Analysis

[ Slides ]

Topics:

Statistical approaches
Structural approaches

Readings:

SS Ch 7
GW Sec 11.3.3

Image Segmentation

[ Slides ]

Topics:

Histogram-based approaches
Clustering-based approaches
Region growing
Split-and-merge
Morphological approaches
Graph-based approaches

Readings:

SS Ch 10.1
GW Ch 10.4-10.5

References:

J. Shi, J. Malik, "Normalized Cuts and Image Segmentation" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(8):888-905, August 2000.
C. Carson, S. Belongie, H. Greenspan, J. Malik, "Blobworld: Image Segmentation Using Expectation-Maximization and Its Applications to Image Querying" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 24(8):1026-1038, August 2002.
Y. Deng, B. S. Manjunath, "Unsupervised Segmentation of Color-Texture Regions in Images and Video" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(8):800-810, August 2001.
P. Soundararajan, S. Sarkar, "An In-Depth Study of Graph Partitioning Measures for Perceptual Organization" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(6):642-660, June 2003.
Image segmentation research at UC Santa Barbara
Graph-based image segmentation tutorial by Shi, Fowlkes, Martin, Sharon at CVPR 2004
L. Vincent, P. Soille, "Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(6):583-598, June 1991.
C. Vachier, F. Meyer, "The Viscous Watershed Transform" (local copy), Journal of Mathematical Imaging and Vision, 22(2-3):251-267, 2005.
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, "SLIC Superpixels Compared to State-of-the-art Superpixel Methods" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 34(11):2274-2282, May 2012.

Software:

Representation and Description

[ Slides ]

Topics:

Image representations and descriptors
Region representations and descriptors

Readings:

SS Ch 10.2, 3.7
GW Ch 11

References:

D. Zhang, G. Lu, "Review of Shape Representation and Description Techniques" (local copy), Pattern Recognition, 37(1):1-19, January 2004.

Pattern Recognition Overview

[ Slides: Part 1 | Part 2 ]

Topics:

Brief introduction to pattern recognition

Readings:

SS Ch 4
GW Ch 12.1-12.2

References:

A. K. Jain, R. P. W. Duin, J. Mao, "Statistical Pattern Recognition: A Review" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(1):4-37, January 2000.

Software:

K-means clustering demo by Yi Li

Case Studies

[ Slides: Part 1 | Part 2 ]

Topics:

Image retrieval
Image classification
Object recognition

References:

Y. Liu, D. Zhang, G. Lu, W.-Y. Ma, "A Survey of Content-Based Image Retrieval with High-Level Semantics" (local copy), Pattern Recognition, 40(1):262-282, January 2007.
X. S. Zhou, T. S. Huang, "Relevance Feedback in Image Retrieval: A Comprehensive Review" (local copy), Multimedia Systems, 8(6):536-544, April 2003.
S. Antani, R. Kasturi, R. Jain, "A Survey on the Use of Pattern Recognition Methods for Abstraction, Indexing and Retrieval of Images and Video" (local copy), Pattern Recognition, 35(4):945-965, April 2002.
A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain, "Content-Based Image Retrieval at the End of the Early Years" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1349-1380, December 2000.
Y. Rui, T. S. Huang, S.-F. Chang, "Image Retrieval: Current Techniques, Promising Directions, and Open Issues" (local copy), Journal of Visual Communication and Image Representation, 10(1):39-62, March 1999.
Y. Rubner, C. Tomasi, L. J. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval" (local copy), International Journal of Computer Vision, 40(2):99-121, November 2000.
A. Vailaya, M. A. T. Figueiredo, A. K. Jain, H.-J. Zhang, "Image Classification for Content-Based Indexing" (local copy), IEEE Trans. on Image Processing, 10(1):117-130, January 2001.
L. Fei-Fei, P. Perona, "A Bayesian Hierarchical Model for Learning Natural Scene Categories" (local copy), IEEE Conference on Computer Vision and Pattern Recognition, 2:524-531, June 20-25, 2005.
P. Quelhas, F. Monay, J.-M. Odobez, D. Gatica-Perez, T. Tuytelaars, "A Thousand Words in a Scene" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 29(9):1575-1589, September 2007.
D. Gokalp, S. Aksoy, "Scene Classification Using Bag-of-Regions Representations", IEEE Conference on Computer Vision and Pattern Recognition, Beyond Patches Workshop, June 23, 2007.
M. R. Boutell, J. Luo, C. M. Brown, "Factor Graphs for Region-Based Whole-Scene Classification" (local copy), IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Semantic Learning Applications in Multimedia, June 17, 2006.
Y. Li, J. A. Bilmes, L. G. Shapiro, "Object Class Recognition using Images of Abstract Regions" (local copy), IAPR International Conference on Pattern Recognition, 1:40-43, August 23-26, 2004.
Y. Li, L. G. Shapiro, J. A. Bilmes, "A Generative/Discriminative Learning Algorithm for Image Classification" (local copy), IEEE International Conference on Computer Vision, 2:1605-1612, October 17-21, 2005.
Recognizing and learning object categories tutorial by Fei-Fei, Fergus and Torralba at ICCV 2009
Visual object recognition tutorial by Leibe and Grauman at AAAI 2008 (local copy)

Exams

The midterm exam will be held at EB 101 and 102 during 17:45-19:45 on April 15, 2015. The exam will cover all topics from the beginning of the semester until the end of the representation and description chapter. You are allowed to bring only the lecture notes (slides). Exam locations are as follows (both classrooms are in the Engineering Building):
- EB 101: AKDEMIR - NGUYEN
- EB 102: ONCUL - YILMAZ
There will be several pop-up quizzes throughout the semester. There will be no make-up quiz for the missed quizzes.

Homework

Homework assignment 1: description | data (Due: March 12, 2015 as online submission)
Homework assignment 2: description | data (Due: April 9, 2015 as online submission)
Homework assignment 3: description | data (Due: May 3, 2015 as online submission)

Please make sure you fully understand the honor code in the syllabus as well as the Bilkent University Policy on Academic Honesty (in Turkish) and the Rules and Regulations of the Higher Education Council (YOK) (in Turkish). Cheating and plagiarism on exams, quizzes, and assignments will be punished according to these regulations.

Project

The goal of the project is to develop an object recognition system based on the bag-of-words model.

Project description and data are available.
You must submit the final report and the developed code as two separate files (a pdf file for the report and a single archive file (e.g., zip, tar, rar) for the code) using the online form by 23:59 on May 21, 2015. No late submission is allowed for project reports.
The reports are expected to be around 6-8 pages and must follow the IEEE two-column format as described in their templates. Try to follow the format as closely as possible.

Grading Policy

Homework:	35%
Quiz:	10%
Exam:	25%
Project:	25%
Class participation:	5%

Announcements

Personnel

Course Information

Texts

Lectures

Topics

Contents

Introduction

Digital Image Fundamentals

Binary Image Analysis

Linear Filtering

Edge Detection

Local Feature Detectors

Color Image Processing

Texture Analysis

Image Segmentation

Representation and Description

Pattern Recognition Overview

Case Studies

Exams

Homework

Project

Grading Policy

Related Links