1. (Jan 29) Syllabus is available.
  2. (Jan 30) Slides for Introduction to Pattern Recognition are available.
  3. (Feb 4) Slides for Bayesian Decision Theory are available.
  4. (Feb 8) Lecture schedule is updated.
  5. (Feb 14) First part of the slides for Parametric Models is available.
  6. (Feb 22) Second part of the slides for Parametric Models is available.
  7. (Feb 22) Office hours are updated.
  8. (Feb 23) Homework assignment 1 is available.
  9. (Mar 5) Third part of the slides for Parametric Models is available.
  10. (Mar 13) Fourth part of the slides for Parametric Models is available.
  11. (Mar 19) Slides for Non-parametric Methods are available.
  12. (Mar 21) Updated slides for Hidden Markov Models.
  13. (Mar 21) Homework assignment 2 is available.
  14. (Mar 22) Slides for Feature Reduction and Selection are available.
  15. (Apr 4) First part of the slides for Non-Bayesian Classifiers is available.
  16. (Apr 10) Second part of the slides for Non-Bayesian Classifiers is available.
  17. (Apr 10) There will not be any class during the week of April 17.
  18. (Apr 11) Third part of the slides for Non-Bayesian Classifiers is available.
  19. (Apr 23) Slides for Unsupervised Learning and Clustering are available.
  20. (Apr 26) Homework assignment 3 is available.
  21. (May 1) Due date for project progress reports is postponed to May 8th, and due date for homework 3 is postponed to May 12th.
  22. (May 2) Grades for the first two quizzes are available.
  23. (May 2) Slides for Algorithm-Independent Learning Issues are available.
  24. (May 9) Slides for Structural and Syntactic Pattern Recognition are available.
  25. (May 14) Updated slides for Structural and Syntactic Pattern Recognition with examples for pattern description using grammars.
  26. (May 18) Grades for the first homework assignment are available.
  27. (May 21) Project final report due date and presentation date are postponed by one day.
  28. (May 24) Grades for the third quiz are available.
  29. (May 24) Poster presentations will be made at the entrance floor of the EB building (in front of the Mithat Coruh auditorium) during 14:30-17:00 on May 25th.
  30. (May 24) You should submit your project software and data using the online form by the end of May 25th.
  31. (May 30) Grades for the second and third homework assignments are available.
  32. Added all projects' final reports.


Instructor: Selim Aksoy
Office: EA 423
Office Hours: Wed 13:40-15:30

Course Information

Schedule: Mon 9:00-10:20, Wed 9:40-11:00 (EA 502)
Mailing List:
Prerequisites: Probability theory, statistics, linear algebra
  • R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000. (required)
  • A. Webb, Statistical Pattern Recognition, Arnold Publishers, 1999.
  • C. M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, 1995.
  • K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, 1990.
  • R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, Inc., 1992.
  • A. K. Jain, R. C. Dubes, Algorithms for Clustering Data, Prentice Hall, 1988.

Lecture Schedule



Introduction to Pattern Recognition

[ Slides ]

(Feb 1, 6)

  • Pattern recognition systems
  • The design cycle
  • An example
  • DHS Ch 1, Appendix A.1-A.2, A.4-A.5

Bayesian Decision Theory

[ Slides ]

(Feb 8, 13, 15)

  • Modeling using continuous and discrete features
  • Discriminant functions
  • The Gaussian density
  • Error estimation
  • DHS Ch 2.1-2.9

Parametric Models

[ Slides: Part 1 | Part 2 | Part 3 | Part 4 ]

(Feb 20, 22, 27, Mar 6, 8, 13, 15)

  • Maximum-likelihood estimation
  • Bayesian estimation
  • Expectation-Maximization and mixture density estimation
  • Hidden Markov Models
  • Bayesian Belief Networks
  • DHS Ch 3.1-3.5, 3.9, 10.2-10.4, 3.10, 2.11

Non-parametric Methods

[ Slides ]

(Mar 20, 22)

  • Density estimation
  • Histogram-based estimation
  • Parzen windows estimation
  • Nearest neighbor estimation
  • DHS Ch 4.1-4.4

Spring Break

(Mar 27-31)

No class

Feature Reduction and Selection

[ Slides ]

(Apr 3)

  • Problems of dimensionality
  • Component analysis
    • Principal components analysis (PCA)
    • Linear discriminant analysis (LDA)
  • Feature selection
  • DHS Ch 3.7-3.8, 10.13-10.14

Non-Bayesian Classifiers

[ Slides: Part 1 | Part 2 | Part 3 ]

(Apr 5, 10, 12, 24)

  • k-nearest neighbor classifier
  • Linear discriminant functions
  • Support vector machines
  • Neural networks
  • Decision trees
  • DHS Ch 4.5-4.6, 5.1-5.3, 5.11, 6.1-6.3, 8.1-8.3

Unsupervised Learning and Clustering

[ Slides ]

(Apr 26, May 1)

  • Criterion functions for clustering
  • k-means clustering
  • Hierarchical clustering
  • Graph-theoretic clustering
  • Cluster validity
  • DHS Ch 10.1, 10.6-10.7, 10.9-10.10, 10.12

Algorithm-Independent Learning Issues

[ Slides ]

(May 3, 8)

  • No Free Lunch Theorem
  • Resampling for classifier design
  • Comparing classifiers
  • Combining classifiers
  • DHS Ch 9.1-9.2, 9.5-9.7

Structural and Syntactic Pattern Recognition

[ Slides ]

(May 10, 15)

  • Recognition with strings
  • Grammatical methods
  • Graph-theoretic methods
  • DHS Ch 8.5-8.6


  1. Homework assignment 1 (Due: March 6 as hardcopy in the class)
  2. Homework assignment 2 (Due: April 3 as online submission)
  3. Homework assignment 3 (Due: May 12 as online submission)

Late submission policy: Unless you make prior arrangements with me (before the due date), no late homework will be accepted.


The purpose of the project is to enable the students to get some hands-on experience in the design, implementation and evaluation of pattern recognition algorithms by applying them to real-world problems. The objective is to try multiple algorithms for different steps of the design cycle such as feature extraction and selection, model learning and estimation, classification and evaluation, to get as high an accuracy as possible on the selected datasets.

You can use your own data from your thesis research, select datasets from the list of data resources below, or contact the instructor for data from ongoing research on multimedia analysis (including video, audio and text data) or image classification. In any case, you should get prior approval before starting your project.

You are free to use any programming language but Matlab is strongly recommended because it is very convenient for prototyping and has many tools available for pattern recognition. You can write the codes yourself or use any code that is available in the public domain. In case you use somebody else's code, you are required to know the details of the algorithms that the code implements.

You are required to work as a group, and submit a project proposal, an interim progress report, a final report written in a conference paper format, and make a poster presentation during the finals week. Tentative schedule of the project is as follows:

All reports and software can be submitted using the online form.

Final report guidelines:

All projects' final reports are available as a pdf file (Bilkent only access).

# Projects Members
1 Object class recognition in natural scenes Kerem Altun, Mert Duatepe
2 Segmentation for image classification and retrieval Demir Gokalp, Turhan O. Daybelge
3 Performance evaluation of various pattern recognition algorithms on disturbance classification for power systems Zeynep Yucel, Erdem Ulusoy
4 Comparison of missing data handling techniques Metin Koc, Metin Tekkalmaz
5 Three-class obstacle classification Oguzcan Oguz, Ahmet Tolgay, Iskender Yakin
6 Information extraction using hidden Markov models Rifat Ozcan, Serhan Tatar, Kerem Ali Ulug
7 Commercial detector in news videos Tolga Can, Esra Ataer
8 Missing feature imputation: comparison of EM and averaging algorithms on arrhythmia data Aysen Tunca, Bayram Boyraz
9 Make and model recognition of a car Rifat Aras, Barkin Basarankut, Hasan Hakan Ari
10 Adjacency modeling in geospatial objects using hidden Markov model Hayati Cam, Ozge Cavus, Emel Kaya

Grading Policy

Homework and quiz:55%
Term project:40%
Class participation:5%

All grades assigned so far are available as a pdf file.

Related Links