Announcements

  1. (Feb 1) Syllabus and slides for Introduction to Pattern Recognition are available.
  2. (Feb 8) Slides for Bayesian Decision Theory are available.
  3. (Feb 10) Added links to software resources.
  4. (Feb 17) First part of the slides for Parametric Models is available.
  5. (Feb 22) Added details about the term project and links to data resources.
  6. (Feb 22) Added links to Matlab tutorials.
  7. (Feb 23) Homework assignment 1 is available.
  8. (Feb 24) Second part of the slides for Parametric Models is available.
  9. (Feb 28) Second part of the slides for Parametric Models is updated with examples.
  10. (Mar 3) The first draft of the third part of the slides for Parametric Models is available.
  11. (Mar 7) Final version of the third part of the slides for Parametric Models is available.
  12. (Mar 10) The first draft of the fourth part of the slides for Parametric Models is available.
  13. (Mar 11) Final version of the fourth part of the slides for Parametric Models is available.
  14. (Mar 15) Homework assignment 2 is available.
  15. (Mar 21) Slides for Non-parametric Methods are available.
  16. (Mar 23) Homework assignment 2 deadline is postponed to March 28.
  17. (Mar 28) Slides for Feature Reduction and Selection are available.
  18. (Mar 30) Homework assignment 3 is available.
  19. (Mar 31) First part of the slides for Non-Bayesian Classifiers is available.
  20. (Apr 5) Second part of the slides for Non-Bayesian Classifiers is available.
  21. (Apr 7) Project proposals can be submitted using the online form.
  22. (Apr 19) Third part of the slides for Non-Bayesian Classifiers is available.
  23. (Apr 24) Slides for Unsupervised Learning and Clustering are available.
  24. (Apr 24) There are no classes on April 26 and 28 but there is a make-up class at EB 202 between 18:00-20:00 on April 25.
  25. (May 2) Homework assignment 4 is available.
  26. (May 9) Slides for Algorithm-Independent Learning Issues are available.
  27. (May 17) Slides for Structural and Syntactic Pattern Recognition are available.
  28. (May 19) Added all quiz grades.
  29. (May 20) Added project final report guidelines.
  30. (May 23) Poster presentations will be made at the entrance floor of the EB building (in front of the Mithat Coruh auditorium) during 14:30-16:30 on May 25th.
  31. (May 28) Added grades for homework assignments 2 and 3.
  32. (May 29) Added grades for project and homework assignment 4.
  33. (May 30) Added grades for homework assignment 1.
  34. (Jun 29) Added all projects' final reports.

Personnel

Instructor: Selim Aksoy
Office: EA 423
Email:
Office Hours: Thu 11:40-12:30

Course Information

Schedule: Tue 8:40-10:30, Thu 10:40-11:30 (EB 204)
Mailing List: http://retina.cs.bilkent.edu.tr/mailman/listinfo/cs551-spring2005
Prerequisites: Probability theory, statistics, linear algebra
Texts:
  • R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000. (required)
  • C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.
  • K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, 1990.
  • R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, Inc., 1992.
  • A. Webb, Statistical Pattern Recognition, Arnold Publishers, 1999.
  • A. K. Jain, R. C. Dubes, Algorithms for Clustering Data, Prentice Hall, 1988.

Lecture Schedule

Chapters

Contents

Introduction to Pattern Recognition

[ Slides ]

(Feb 3)

Topics:
  • Pattern recognition systems
  • The design cycle
  • An example
Readings:
  • DHS Ch 1, Appendix A.1-A.2, A.4-A.5
References:

Bayesian Decision Theory

[ Slides ]

(Feb 8, 10, 15, 17)

Topics:
  • Modeling using continuous and discrete features
  • Discriminant functions
  • The Gaussian density
  • Error estimation
Readings:
  • DHS Ch 2.1-2.9

Parametric Models

[ Slides: Part 1 | Part 2 | Part 3 | Part 4 ]

(Feb 22, 24, Mar 1, 3, 8, 10)

Topics:
  • Maximum-likelihood estimation
  • Bayesian estimation
  • Expectation-Maximization and mixture density estimation
  • Hidden Markov Models
  • Bayesian Belief Networks
Readings:
  • DHS Ch 3.1-3.5, 3.9, 10.2-10.4, 3.10, 2.11
References:

Non-parametric Methods

[ Slides ]

(Mar 15, 17)

Topics:
  • Density estimation
  • Parzen windows estimation
  • Nearest neighbor estimation
Readings:
  • DHS Ch 4.1-4.4

Feature Reduction and Selection

[ Slides ]

(Mar 22, 24)

Topics:
  • Problems of dimensionality
  • Component analysis
    • Principal components analysis (PCA)
    • Linear discriminant analysis (LDA)
  • Feature selection
Readings:
  • DHS Ch 3.7-3.8, 10.13-10.14
References:

Non-Bayesian Classifiers

[ Slides: Part 1 | Part 2 | Part 3 ]

(Mar 29, 31, Apr 5, 7)

Topics:
  • k-nearest neighbor classifier
  • Linear discriminant functions
  • Support vector machines
  • Neural networks
  • Decision trees
Readings:
  • DHS Ch 4.5-4.6, 5.1-5.3, 5.11, 6.1-6.3, 8.1-8.3
References:

Spring Break

(Apr 11-15)

No class

Unsupervised Learning and Clustering

[ Slides ]

(Apr 19, 21, 26)

Topics:
  • Criterion functions for clustering
  • k-means clustering
  • Hierarchical clustering
  • Graph-theoretic clustering
  • Cluster validity
Readings:
  • DHS Ch 10.1, 10.6-10.7, 10.9-10.10, 10.12
References:

Algorithm-Independent Learning Issues

[ Slides ]

(Apr 28, May 3, 5, 10)

Topics:
  • No Free Lunch Theorem
  • Resampling for classifier design
  • Comparing classifiers
  • Combining classifiers
Readings:
  • DHS Ch 9.1-9.2, 9.5-9.7
References:

Structural and Syntactic Pattern Recognition

[ Slides ]

(May 12, 17)

Topics:
  • Recognition with strings
  • Grammatical methods
  • Graph-theoretic methods
Readings:
  • DHS Ch 8.5-8.6
References:
  • K.-S. Fu, T. L. Booth, "Grammatical Inference: Introduction and Survey - Part I," IEEE Trans. on Pattern Analysis and Machine Intelligence, 8(3):343-359, 1986.
  • K.-S. Fu, T. L. Booth, "Grammatical Inference: Introduction and Survey - Part II," IEEE Trans. on Pattern Analysis and Machine Intelligence, 8(3):360-375, 1986.
  • L. G. Shapiro, R. M. Haralick, "A Metric for Comparing Relational Descriptions," IEEE Trans. on Pattern Analysis and Machine Intelligence, 7:90-94, 1985.
  • B. T. Messmer, H. Bunke, "Efficient Subgraph Isomorphism Detection: A Decomposition Approach" (local copy), IEEE Trans. on Knowledge and Data Engineering, 12(2):307-323, March/April 2000.
  • W. J. Christmas, J. Kittler, M. Petrou, "Structural Matching in Computer Vision Using Probabilistic Relaxation" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 17(8):749-764, August 1995.
  • R. Myers, R. C. Wilson, E. R. Hancock, "Bayesian Graph Edit Distance" (local copy), IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(6):628-635, June 2000.

Assignments

  1. Homework assignment 1 (Due: March 3 as hardcopy in the class)
  2. Homework assignment 2 (Due: March 28 as online submission)
  3. Homework assignment 3 (Due: April 18 as online submission)
  4. Homework assignment 4 (Due: May 13 as online submission)

Late submission policy: Unless you make prior arrangements with me (before the due date), your grade will be multiplied by "1 - 0.33(t-due)" where due is the assignment due date and t is the date you submit it (t=due, due+1, due+2).

Project

The purpose of the project is to enable the students to get some hands-on experience in the design, implementation and evaluation of pattern recognition algorithms by applying them to real-world problems. The objective is to try multiple algorithms for different steps of the design cycle such as feature extraction and selection, model learning and estimation, classification and evaluation, to get as high an accuracy as possible on the selected datasets.

You can use your own data from your thesis research, select datasets from the list of data resources below, or contact the instructor for data from ongoing research on multimedia analysis (including video, audio and text data) or image classification. In any case, you should get prior approval before starting your project.

You are free to use any programming language but Matlab is strongly recommended because it is very convenient for prototyping and has many tools available for pattern recognition. You can write the codes yourself or use any code that is available in the public domain. In case you use somebody else's code, you are required to know the details of the algorithms that the code implements.

You are required to work in groups of three, and submit a project proposal, an interim progress report, a final report written in a conference paper format, and make a poster presentation during the finals week. Tentative schedule of the project is as follows:

All reports and software can be submitted using the online form.

Final report guidelines:

All projects' final reports are available as a pdf file (Bilkent only access).

# Projects Members
1 Multi-scale region-based remote sensing image classification using a Hidden Markov Model Gokhan Akcay, Bayram Boyraz, Derya Ozkan
2 Segmentation algorithm for news videos based on speech / speech+music classification Caglar Ari, Osman Tapkan
3 A probabilistic content model learning application Ibrahim Demir, Gonenc Ercan, E. Kartal Tabak
4 Region based object recognition using boosting algorithm Demir Gokalp
5 Offline handwritten word recognition Ayca Ozcelikkale, Deniz Ustebay, Funda Durupinar
6 Comparison of unsupervised learning algorithms Hande Dogan, Sami Ezercan
7 Comparison of feature sets for text categorization Yigithan Dedeoglu, I. Emre Sahin, H. Ozgur Tan, Yasin Uzun
8 Frontal face detection and recognition A. Polat Ay, Kivanc Kose, Ayse Kucukyilmaz, Onur Onder
9 Comparison of classification algorithms using semi-supervised learning on middle-scale datasets M. Erol Aran, Faysal Basci, H. Dogu Taskiran
10 A performance analysis of various pattern recognition algorithms on hazelnut impact sounds Berkan Dulek, Ibrahim Onaran, Mehmet Turkan
11 Discrimination between upstream and downstream genes Ozgun Babur
12 A comparative study of different techniques to target differentiation and localization with infrared sensors Cagri Yuzbasioglu, Ilker Bayram, Serkan Onart, Tayfun Aytac

Grading Policy

Homework and quiz:55%
Term project:40%
Class participation:5%

Grades (Bilkent only access) for all quizzes, homework assignments and project are available (also available on SAPS).

Related Links