Personnel
Instructor:  Selim Aksoy 

Office:  EA 422 
Email: 
Course Information
Schedule:  Tue 13:4015:30, Thu 15:4017:30 (EA 502) 

Office hours:  TBD (EA 422) 
Prerequisites:  Probability theory, statistics, linear algebra 
Texts
 K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
 C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
 R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000.
 S. Theodoridis, K. Koutroumbas, Pattern Recognition, 3rd edition, Academic Press, 2006.
 D. Koller, N. Friedman, Probabilistic Graphical Models: Principals and Techniques, MIT Press, 2009.
 A. Webb, Statistical Pattern Recognition, 2nd edition, John Wiley & Sons, Inc., 2002.
 T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning, Springer, 2003.
 K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, 1990.
 R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, Inc., 1992.
 A. K. Jain, R. C. Dubes, Algorithms for Clustering Data, Prentice Hall, 1988.
Lecture Schedule
Chapters 
Contents 

Introduction to Pattern Recognition[ Slides ] (Sep 16, 18) 
Topics:
References:

Bayesian Decision Theory[ Slides ] (Sep 23, 25, 30) 
Topics:
References:

Parametric Models[ Slides: Part 1  Part 2  Part 3 ] (Sep 30, Oct 2, 9, 14, 16, 21, 23) 
Topics:
References:

Nonparametric Methods(Oct 28, 30) 
Topics:

Probabilistic Graphical Models(Nov 4, 6, 18, 20) 
Topics:
References:

Feature Reduction and Selection(Nov 25, 27) 
Topics:
References:

NonBayesian Classifiers(Dec 2, 4) 
Topics:
References:

Structural and Syntactic Pattern Recognition(Dec 9, 11, 18, 23, 30) 
Topics:
Readings:
References:

Exam
 Midterm exam will be held at EA 502 during 13:4015:30 (class hours) on December 16, 2014. The exam will cover all topics from the beginning of the semester until the end of the nonBayesian classifiers chapter.
Assignments
 Homework assignment 1: description (Due: November 3, 2014 as online submission)
 Homework assignment 2
 Homework assignment 3
Late submission policy: Unless you make prior arrangements with me (before the due date), no late homework will be accepted.
Project
The purpose of the project is to enable the students to get some handson experience in the design, implementation and evaluation of pattern recognition algorithms by applying them to realworld problems. The objective is to try multiple algorithms for different steps of the design cycle such as feature extraction and selection, model learning and estimation, classification and evaluation, to get an acceptable accuracy on the selected data sets. An acceptable accuracy will be problem dependent but needs to be measured using formal quantitative methods.
You can use your own data from your thesis research, select data sets from the list of data resources below, or contact the instructor. In any case, you should get prior approval before starting your project.
You are free to use any programming language but Matlab is strongly recommended because it is very convenient for prototyping and has many tools available for pattern recognition. You can write the codes yourself or use any code that is available in the public domain. In case you use somebody else's code, you are required to properly cite its source and know the details of the algorithms that the code implements.
You are required to work in groups of two, and submit a project proposal, an interim progress report, and a final report written in a conference paper format. Tentative schedule of the project is as follows:
 Project proposal (due December 1, 2014): Submit a 12 page proposal that describes the problem you would like to tackle, objective of the study, proposed algorithms, hardware/software tools and data that you plan to utilize, and evaluation strategies that you plan to use. Also provide a short list of related references.
 Interim progress report (due December 22, 2014): Submit a report that describes your progress with the project and your plans for the rest of the semester.
 Final report (due January 5, 2014): Submit a readable and wellorganized report that provides proper motivation for the task, proper citation and discussion of related literature, proper explanation of the details of the approach and implementation strategies, proper performance evaluation, and detailed discussion of the results. Highlight your contributions and conclusions as a team. Also submit welldocumented software with your report. Each team member should also provide a written description of her/his own contributions to the project.
All reports and software can be submitted using the online form.
Final report guidelines:
 The reports are expected to be 6 pages and must follow the IEEE Computer Society twocolumn format as described in their templates. Try to follow the format as closely as possible.
 PDF submission is required.
Grading Policy
Midterm exam:  25% 
Homework:  40% 
Project:  30% 
Class participation:  5% 
Related Links
 Previous semesters for CS 551
 Murphy book
 Bishop book
 Duda, Hart, Stork book
 Book's website
 Make sure you check the errata for the particular printing you have.
 Theodoridis and Koutroumbas book
 Koller and Friedman book
 Webb book
 Hastie, Tibshirani, Friedman book
 Software resources
 PRTools by the Delft Pattern Recognition Group (in Matlab) (local copy)
 Matlab Toolbox for Dimensionality Reduction
 Netlab Neural Network Software (in Matlab) (local copy of software and its documentation)
 Weka Data Mining Software (in Java)
 Bayes Net Toolbox (in Matlab)
 Hidden Markov Model Toolbox (in Matlab)
 SVMlight  SVM training package (in C)
 Sequential Minimal Optimization algorithm for SVM training
 LIBSVM  A Library for SVM (in C++ and Java, with interfaces for additional languages)
 Numerical Recipes (in C)
 Software resources from Pattern Recognition Information web site
 Software resources from Kevin Murphy's web site
 Software resources from Kernel Machines web site
 Software resources from Kernel Methods web site
 Software resources from American Association for Artificial Intelligence web site
 StatLib
 Mathtools.net Technical Computing (in Matlab, C/C++, Java)
 Matlab tutorials
 Data resources
 Pattern recognition related archives
 Computer vision test images
 UCI Machine Learning Repository
 Labeled databases for object detection
 Image database from the University of Washington
 Texture database from the University of Oulu
 Document database from the University of Oulu
 Other databases from the University of Oulu
 Image databases from CMU Vision and Autonomous Systems Center
 Various other data sets from the University of Washington
 Face databases from CMU
 Face databases from MIT
 Another page for face databases
 MNIST Database of handwritten digits
 Shape database from Brown University
 Reuters21578 Text Categorization Collection
 NIST Scientific and Technical Databases
 RISC: Repository of Information on Semisupervised Clustering
 Others
 Pattern Recognition Information
 International Association for Pattern Recognition (IAPR)
 IEEE Computer Society Technical Committee on Pattern Analysis and Machine Intelligence (PAMI)
 IAPR Education Committee Resources (Tutorials, data sets, codes, etc.)
 IAPR Technical Committee 1 on Statistical Techniques in Pattern Recognition
 IAPR Technical Committee 2 on Structural and Syntactical Pattern Recognition
 MathWorld (an online encyclopedia of mathematical resources)
 International Society for Bayesian Analysis
 Statistical Learning/Pattern Recognition Glossary
 Statistical Data Mining Tutorials
 Kernel Machines
 Learning with Kernels
 Engineering Statistics Handbook
 Introductory Statistics: Concepts, Models, and Applications
 The Probability Web