Announcements

  1. (Feb 8) Course page is online.
  2. (Feb 8) Syllabus is available.
  3. (Feb 8) Slides for Introduction to Pattern Recognition are available.
  4. (Feb 15) Slides for Bayesian Decision Theory are available.
  5. (Feb 22) First part of the slides for Parametric Models is available.
  6. (Mar 1) Second part of the slides for Parametric Models is available.
  7. (Mar 3) Homework assignment 1 is available.
  8. (Mar 9) Third part of the slides for Parametric Models is available.
  9. (Mar 14) Fourth part of the slides for Parametric Models is available.
  10. (Mar 17) Homework assignment 2 is available.
  11. (Mar 22) Slides for Non-parametric Methods are available.
  12. (Mar 29) Slides for Feature Reduction and Selection are available.
  13. (Apr 5) First part of the slides for Non-Bayesian Classifiers is available.
  14. (Apr 5) Second part of the slides for Non-Bayesian Classifiers is available.
  15. (Apr 13) Third part of the slides for Non-Bayesian Classifiers is available.
  16. (Apr 14) Slides for Unsupervised Learning and Clustering are available.
  17. (Apr 26) Slides for Algorithm-Independent Learning Issues are available.
  18. (Apr 26) Homework assignment 3 is available.
  19. (May 3) Slides for Structural and Syntactic Pattern Recognition are available.

Personnel

Instructor: Selim Aksoy
Office: EA 423
Email:

Course Information

Schedule: Mon 8:40-10:30, Wed 10:40-11:30 (EA 502)
Office hours: TBD
Prerequisites: Probability theory, statistics, linear algebra

Texts

Lecture Schedule

Chapters

Contents

Introduction to Pattern Recognition

[ Slides ]

(Feb 9, 11)

Topics:
  • Pattern recognition systems
  • The design cycle
  • An example
Readings:
  • DHS Ch 1, Appendix A.1-A.2, A.4-A.5
References:

Bayesian Decision Theory

[ Slides ]

(Feb 16, 18, 23)

Topics:
  • Modeling using continuous and discrete features
  • Discriminant functions
  • The Gaussian density
  • Error estimation
Readings:
  • DHS Ch 2.1-2.9

Parametric Models

[ Slides: Part 1 | Part 2 | Part 3 | Part 4 ]

(Feb 25, Mar 2, 4, 9, 11, 16, 18)

Topics:
  • Maximum-likelihood estimation
  • Bayesian estimation
  • Expectation-Maximization and mixture density estimation
  • Hidden Markov Models
  • Bayesian Belief Networks
Readings:
  • DHS Ch 3.1-3.5, 3.9, 10.2-10.4, 3.10, 2.11
References:

Non-parametric Methods

[ Slides ]

(Mar 23, 25)

Topics:
  • Density estimation
  • Histogram-based estimation
  • Parzen windows estimation
  • Nearest neighbor estimation
Readings:
  • DHS Ch 4.1-4.4

Feature Reduction and Selection

[ Slides ]

(Mar 30, Apr 1)

Topics:
  • Problems of dimensionality
  • Component analysis
    • Principal components analysis (PCA)
    • Linear discriminant analysis (LDA)
  • Feature selection
Readings:
  • DHS Ch 3.7-3.8, 10.13-10.14
References:

Non-Bayesian Classifiers

[ Slides: Part 1 | Part 2 | Part 3 ]

(Apr 6, 8, 13)

Topics:
  • k-nearest neighbor classifier
  • Linear discriminant functions
  • Support vector machines
  • Neural networks
  • Decision trees
Readings:
  • DHS Ch 4.5-4.6, 5.1-5.3, 5.11, 6.1-6.3, 8.1-8.3
References:

Unsupervised Learning and Clustering

[ Slides ]

(Apr 15, 20, 22)

Topics:
  • Criterion functions for clustering
  • k-means clustering
  • Hierarchical clustering
  • Graph-theoretic clustering
  • Cluster validity
Readings:
  • DHS Ch 10.1, 10.6-10.7, 10.9-10.10, 10.12
References:

Algorithm-Independent Learning Issues

[ Slides ]

(Apr 27, 29)

Topics:
  • No Free Lunch Theorem
  • Resampling for classifier design
  • Comparing classifiers
  • Combining classifiers
Readings:
  • DHS Ch 9.1-9.2, 9.5-9.7
References:

Structural and Syntactic Pattern Recognition

[ Slides ]

(May 4, 6, 11, 13)

Topics:
  • Recognition with strings
  • Grammatical methods
  • Graph-theoretic methods
Readings:
  • DHS Ch 8.5-8.6
References:

Exams

Assignments

  1. Homework assignment 1 (Due: March 18, 2009 as hardcopy in the class)
  2. Homework assignment 2 (Due: April 8, 2009 as online submission)
  3. Homework assignment 3 (Due: May 24, 2009 as online submission)

Late submission policy: Unless you make prior arrangements with me (before the due date), no late homework will be accepted.

Grading Policy

Homework:50%
Midterm exam:20%
Final exam:25%
Class participation:5%

Related Links