|Schedule:||Tue 13:40-15:30, Thu 15:40-17:30 (EA 502)|
|Office hours:||TBD (EA 422)|
|Prerequisites:||Probability theory, statistics, linear algebra|
- K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, 2012.
- C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
- R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, 2nd edition, John Wiley & Sons, Inc., 2000.
- S. Theodoridis, K. Koutroumbas, Pattern Recognition, 3rd edition, Academic Press, 2006.
- D. Koller, N. Friedman, Probabilistic Graphical Models: Principals and Techniques, MIT Press, 2009.
- A. Webb, Statistical Pattern Recognition, 2nd edition, John Wiley & Sons, Inc., 2002.
- T. Hastie, R. Tibshirani, J. Friedman, The Elements of Statistical Learning, Springer, 2003.
- K. Fukunaga, Introduction to Statistical Pattern Recognition, Academic Press, 1990.
- R. Schalkoff, Pattern Recognition: Statistical, Structural and Neural Approaches, John Wiley & Sons, Inc., 1992.
- A. K. Jain, R. C. Dubes, Algorithms for Clustering Data, Prentice Hall, 1988.
Introduction to Pattern Recognition
[ Slides ]
(Feb 6, 11)
Bayesian Decision Theory
[ Slides ]
(Feb 13, 18, 20)
(Feb 25, 27, Mar 4, 11, 13, 18)
Probabilistic Graphical Models
(Mar 25, 27, Apr 1, 3)
Feature Reduction and Selection
(Apr 8, 10, 15)
Structural and Syntactic Pattern Recognition
(April 22, 24, May 6, 8, 13, 15)
- Midterm exam will be held at EA 502 during 13:40-15:30 (class hours) on April 29, 2014. The exam will cover all topics from the beginning of the semester until the end of the non-Bayesian classifiers chapter.
- Homework assignment 1: description (Due: March 24, 2014 as online submission)
- Homework assignment 2
- Homework assignment 3
Late submission policy: Unless you make prior arrangements with me (before the due date), no late homework will be accepted.
The purpose of the project is to enable the students to get some hands-on experience in the design, implementation and evaluation of pattern recognition algorithms by applying them to real-world problems. The objective is to try multiple algorithms for different steps of the design cycle such as feature extraction and selection, model learning and estimation, classification and evaluation, to get an acceptable accuracy on the selected data sets. An acceptable accuracy will be problem dependent but needs to be measured using formal quantitative methods.
You can use your own data from your thesis research, select data sets from the list of data resources below, or contact the instructor. In any case, you should get prior approval before starting your project.
You are free to use any programming language but Matlab is strongly recommended because it is very convenient for prototyping and has many tools available for pattern recognition. You can write the codes yourself or use any code that is available in the public domain. In case you use somebody else's code, you are required to properly cite its source and know the details of the algorithms that the code implements.
You are required to work in groups of two, and submit a project proposal, an interim progress report, a final report written in a conference paper format, and make a presentation during the finals week. Tentative schedule of the project is as follows:
- Project proposal (due April 14, 2014): Submit a 1-2 page proposal that describes the problem you would like to tackle, objective of the study, proposed algorithms, hardware/software tools and data that you plan to utilize, and evaluation strategies that you plan to use. Also provide a short list of related references.
- Interim progress report (due May 5, 2014): Submit a report that describes your progress with the project and your plans for the rest of the semester.
- Final report (due May 22, 2014): Submit a readable and well-organized report that provides proper motivation for the task, proper citation and discussion of related literature, proper explanation of the details of the approach and implementation strategies, proper performance evaluation, and detailed discussion of the results. Highlight your contributions and conclusions. Also submit well-documented software with your report.
- Presentation (due May 26, 2014): Make a ~10 minute presentation of your work to the class. Each student is expected to attend all presentations. Each team member should also provide a written description of her/his own contributions to the project.
All reports and software can be submitted using the online form.
Final report guidelines:
- The reports are expected to be 6 pages and must follow the IEEE Computer Society two-column format as described in their templates. Try to follow the format as closely as possible.
- PDF submission is required.
- Previous semesters for CS 551
- Duda, Hart, Stork book
- Theodoridis and Koutroumbas book
- Bishop book
- Koller and Friedman book
- Webb book
- Hastie, Tibshirani, Friedman book
- Software resources
- PRTools by the Delft Pattern Recognition Group (in Matlab) (local copy)
- Matlab Toolbox for Dimensionality Reduction
- Netlab Neural Network Software (in Matlab) (local copy of software and its documentation)
- Weka Data Mining Software (in Java)
- Bayes Net Toolbox (in Matlab)
- Hidden Markov Model Toolbox (in Matlab)
- SVMlight - SVM training package (in C)
- Sequential Minimal Optimization algorithm for SVM training
- LIBSVM - A Library for SVM (in C++ and Java, with interfaces for additional languages)
- Numerical Recipes (in C)
- Software resources from Pattern Recognition Information web site
- Software resources from Kevin Murphy's web site
- Software resources from Kernel Machines web site
- Software resources from Kernel Methods web site
- Software resources from American Association for Artificial Intelligence web site
- Mathtools.net Technical Computing (in Matlab, C/C++, Java)
- Matlab tutorials
- Data resources
- Pattern recognition related archives
- Computer vision test images
- UCI Machine Learning Repository
- Labeled databases for object detection
- Image database from the University of Washington
- Texture database from the University of Oulu
- Document database from the University of Oulu
- Other databases from the University of Oulu
- Image databases from CMU Vision and Autonomous Systems Center
- Various other data sets from the University of Washington
- Face databases from CMU
- Face databases from MIT
- Another page for face databases
- MNIST Database of handwritten digits
- Shape database from Brown University
- Reuters-21578 Text Categorization Collection
- NIST Scientific and Technical Databases
- RISC: Repository of Information on Semi-supervised Clustering
- Pattern Recognition Information
- International Association for Pattern Recognition (IAPR)
- IEEE Computer Society Technical Committee on Pattern Analysis and Machine Intelligence (PAMI)
- IAPR Education Committee Resources (Tutorials, data sets, codes, etc.)
- IAPR Technical Committee 1 on Statistical Techniques in Pattern Recognition
- IAPR Technical Committee 2 on Structural and Syntactical Pattern Recognition
- MathWorld (an online encyclopedia of mathematical resources)
- International Society for Bayesian Analysis
- Statistical Learning/Pattern Recognition Glossary
- Statistical Data Mining Tutorials
- Kernel Machines
- Learning with Kernels
- Engineering Statistics Handbook
- Introductory Statistics: Concepts, Models, and Applications
- The Probability Web