Instructor: Pinar Duygulu
Office :  EA 433
e-mail : duygulu[at]cs.bilkent.edu.tr
Phone :  (312) 290 31 43
Office hours:  Tuesdays 13:40-15:30 and by appointment..
Textbook: Computer Vision - A modern Aproach  by David A. Forsyth & Jean Ponce, Prentice Hall, Ed. 1, 2002.  
Related Material: http://www.cs.bilkent.edu.tr/~duygulu/CVlinks.html
    Also other complementary articles that will be made available
    Previous course material is avilable at http://www.cs.bilkent.edu.tr/~duygulu/Courses/CS554/Spring2004
Course web page: http://www.cs.bilkent.edu.tr/~duygulu/Courses/CS554/Fall2004
Time & Location: Thursdays  8:40-11:30, EA 502
Course Description:  Basic concepts in computational vision. Relation to human visual perception. The analysis and understanding of image and video data. Mathematical foundations, image formation and representation, segmentation, feature extraction, contour and region analysis, camera geometry and calibration, stereo, motion, 3-D reconstruction, object and scene recognition, object and people tracking, human activity recognition and inference.
Prerequisites:Knowledge of linear algebra and calculus, probability and statistics
Grading:
        Homeworks 45%  (There will be three programming assignments and two or three reading assignments)
        Paper Presentations 15%
        Term Project 35%
        Class Participation 5%

 

---

Project reports and presentations are available here

---

Announcements:

• Check the latest announcements about the course here at least once a week

 Important notes about evaluation:

Projects 

The project may be

• An original implementation of a new or published idea,
• A detailed emprical evaluation and comparison of the existing implementations of two or more methods,

  Possible Project Ideas:

• Associate the words annotated for an image with features extracted around interest points to recognize objects in the images
• Find the best segmentation of images with the help of the association between annotated keywords and image features
  
• Find the moving objects in news videos
  
• Detect the areas with overlay text in the news videos
  
• Translate Ottoman characters into Turkish characters by using the associations between the visual features of Ottoman characters and their Turkish transcripts.

Presentations:

Your presentations will be evaluated according to the following criteria. Please, consider them in preparing your presentations:

• Understanding of the topic - how confident are you with the paper that you present
• Review of the related work - not just mentioning but by reading some of them to understand and relate to your paper
• Giving an overview of the paper -  the main contributions of the paper, and an overview of the approach
• Explaining the details - understanding and explaining the formulas and methods given in the paper
• Presentation - in general how well you are prepared to give the talk
  

Assignments:

• There will be three reading assignments and three programming assignments      
  
• All homeworks are due midnight and will be sent by e-mail
• Late homeworks are not accepted      
• In your e-mail use the following format in the title
• CS554 - Programming assignment # or
• CS554 - Reading assignment #
• Your reading assignments should be submitted in .pdf in the following format
• <name_surname_RA_#>.pdf
• Your programming assignmenments should be sent as a tar ball in the following format
• <name_surname_PA_#>.tar
• Reading assignments will be summary of the given paper. It should be about one page, and you should explain the main contributions and the important points of the proposed methods in your own words. Do not include any figures or formulas. Do not write the values of parameters or thresholds unless they are very important. Assume that you are submitting one or two pages summary of your paper to a conference.

---

Assignments:
 

• Programming Assignment 1
  
• Write a matlab program that produces Gaussian and Laplacian pyramids for a given image
• Images :  (image1.jpg)
• Due : October 24, Sunday
  
• Grading : 6%
  
• Reading Assignment 1
  
• Write a summary of  the following paper
  
• Distinctive Image Features from Scale-Invariant Keypoints, David Lowe, International Journal of Computer Vision, 2004
• Due: October 20, Wednesday
  
• Grading: 3%
  
• Programming assignment 2
• Write a matlab code for feature tracking according to the details explained here (adapted from Martial Hebert)
  
• Due: December 15, Wednesday
  
• Grading: 20%
  
• Programming Assignment 3
  
• Retrieve images from a database for a query image and give rankings
• Extract 64 bit RGB color histogram
• Segment the image into 5 regions (fixed size blocks)
  
• divide the image into 2x2 grids to construct 4 regions,
  
• take another region from the center of the image with the same size as the other  regions.
  
• Compare images using
  
• global color histogram extracted from the whole image
  
• RGB mean and standard deviation values extracted from the regions
• Lab mean and standard deviation values extracted from the regions
• For each query image give the most similar 5 images for each of the methods above, compare the results
• To give a quantitive result, find the number of images from the same CD among those 5 retrieved images
  
• Images :  (hw3-images.zip)
• There are 10 datasets including 6 images each. Query images are as follows, the others will be used as the images to be retrieved.
• 1000.jpg, 10000.jpg, 17036.jpg, 84079.jpg, 107001.jpg, 108019.jpg, 113003.jpg, 135013.jpg, 156004.jpg, 181066.jpg
  
• Due: December 26, Sunday
  
• Grading: 10%

• Reading Assignment 2
  
• Write a summary of  the following paper
• Normalized cuts and image segmentation, Jianbo Shi and Jitendra Malik, IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 2000
• a shorter version  published in IEEE Conf. Computer Vision and Pattern Recognition(CVPR), June 1997, Puerto Rico
• Due: November 24, Wednesday
  
• Grading: 3%
• Reading Assignment 3
  
• Write a summary of the following paper
• Face recognition using eigenfaces, M. Turk and A. Pentland, Proc. IEEE Conference on Computer Vision and Pattern Recognition, Maui, Hawaii, 1991
• Due: December 1, Wednesday
  
• Grading: 3%
  

---

Syllabus

  

. September 23 Introduction	Topics Overview and objectives (slides) Basic linear algebra (slides from Octavia Camps, Penn State) Introduction to Matlab Readings: Matlab primer (.pdf) by Kermit Sigmon, University of Florida Basic math (.pdf),  figures (.pdf) by Martial Hebert, CMU Links The Mathworks' "Getting started with Matlab" Linear Algebra by Jim Hefferon Vision in the news - CNN article The Physics of Art and Visual Perception Optical Illusions The joy of visual perception
. September 30 Color	Topics Light and color (slides) Readings Chapters  4 and 6 Colour in nature, book chapter Is colour constancy good enough?, Brian Funt, Kobus Barnard and Lindsay Martin, 5th European Conference on Computer Vision, 1998, pages 445-459.
. October 7 Liner Filters	Topics Linear filters, convolution, smoothing, derivatives, Fourier transform, sampling and aliazing, gaussian pyramids (slides)  Readings Chapter 7 Computer vision for interactive computer graphics,W. T. Freeman, D. Anderson, P. Beardsley, C. Dodge, H. Kage, K. Kyuma, Y. Miyake, M. Roth, K. Tanaka, C. Weissman, W. Yerazunis,  in IEEE Computer Graphics and Applications, volume 18, number 3, May--June, pp. 42-53, 1998. Links Joy of Convolution, JHU Tutorial on Convolution, by David Young
. October 14 Texture	Topics Texture analysis and synthesis  (slides) Readings Chapter 9 Pyramid based Texture Analysis/Synthesis, David Heeger and James Bergen, SIGGARPH 1995 A Computational Model of Texture Segmentation, J. Malik and P. Perona, Proc. Computer Vision and Pattern Recognition, 1989 Eraly Vision and Texture Perception, J.R. Bergen and E.H. Adelson, Nature, 1988 W.Y. Ma and B.S. Manjunath, Texture features and learning similarity, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 425-430, June, 1996 Links Wavelet Tutorial by Robi Polikar Alyosha Efros' texture synthesis page, graphcut textures, image analogies Presentation by Erol Aran: Texture Synthesis by Non-parametric Sampling, Alexei A. Efros and Thomas K. Leung, IEEE International Conference on Computer Vision (ICCV'99), Corfu, Greece, September 1999,  and Image Quilting for Texture Synthesis and Transfer, Alexei A. Efros and William T. Freeman, Proceedings of SIGGRAPH '01, Los Angeles, California, August, 2001 by Bayram Boyraz:  Is colour constancy good enough?, Brian Funt, Kobus Barnard and Lindsay Martin, 5th European Conference on Computer Vision, 1998, pages 445-459
. October 21 Edge Detection and Interest Points	Edge Detection Topics Derivatives, Edge detection, Hough Transform  (slides) Readings Chapter 8 A Computational Approach to Edge Detection, J. Canny, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 8, No. 6, Nov 1986. Links Tutorial on Hough Transform, by David Young Hough line detection applet, Hough circle detection applet Interest Points Topics Harris Detector, Local invariant points, SIFT  (slides) Readings A combined corner and edge detector, Chris Harris and Mike Stephans, Proceedings of The Fourth Alvey Vision Conference, Manchester, pp 147-151. 1988 Local Greyvalue Invariants for Image Retrieval,  C. Schmid and R. Mohr. In Pattern Analysis and Machine Intelligence, 1997. Indexing based on scale invariant interest points. K.Mikolajczyk and C.Schmid. In International Conference on Computer Vision, 525-531, 2001 Distinctive Image Features from Scale-Invariant Keypoints, David Lowe, International Journal of Computer Vision, 2004. Links Tutorial on Recognition and Matching based on local invariant features, by Cordelia Schmid and David Lowe, at CVPR 2003
October 28	!!!   NO CLASS    !!!
. November 4 Cameras	Topics Camera geometry, Camera calibration (slides) Readings Chapters 1, 2 and 3 An Introduction to Projective Geometry by Stan Birchfield A Flexible New Technique for Camera Calibration, Zhengyou Zhang,  Technical Report MSR-TR-98-71,  Microsoft Research, Redmond, WA 98052 Links Brunelleschi and the Origin of Linear Perspective Links related to Camera Calibration Presentation by Osman Topcu: 3D Object Modeling and Recognition Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints, Fred Rothganger, Svetlana Lazebnik, Cordelia Schmid, and Jean Ponce. International Journal of Computer Vision. (slides)
. November 11 Multi-view geometry &  Stereopsis	Topics Stereopsis, Epipolar geometry, Matching, Reconstruction (slides) Readings Chapters 10 and 11 Epipolar Geometry and the Fundamental Matrix,  in Multiple View Geometry in Computer Vision , Hartley & Zisserman, 2000 Links Tutorials on Multi-view Geometry by Zisserman, Hartley and Pollefeys Middlebury Stereo Vision page - good source for stereo dataset Stereo Images with Ground Truth Disparity and Occlusion, from University of Bonn A Matlab code  by Carlo Tomasi to generate random dot stereograms and to find correspondences with the Marr-Poggio algorithm. The art of Stereo Photography How to freeview Stereo Images  Modeling and Rendering Architecture from Photographs, by Debevec, Taylor and Malik
. November 18 Motion	Topics Optical flow, structure from motion, Tracking (slides) Readings An iterative image registration technique with an application to stereo vision, Bruce Lucas and Takeo Kanade, Proceedings of the 7th International Joint Conference on Artificial Intelligence (IJCAI), 1981 Detection and Tracking of Point Features.Carlo Tomasi and Takeo Kanade.  Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991.    Good Features to Track, Jianbo Shi and Carlo Tomasi, IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994. Feature based methods for structure and motion estimation, Phil Torr and Andrew Zisserman, in Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, R. Szeliski (Eds.), Springer (2000) Links KLT : Implementation of Kanade-Lucas-Tomasi Feature Tracker Motion Flow in Computer Vision, by Jana Kosecka and Shankar Sastry Lecture on motion, by Robyn Owens Layered Image Representation demo by J. Wang Barber pole illusion Image sequences: Flower garden(.mov)
. November 25 Segmentation	Topics Segmentation and Grouping, Line fitting  (slides) Readings Normalized cuts and image segmentation, Jianbo Shi and Jitendra Malik, IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 2000 a shorter version  published in IEEE Conf. Computer Vision and Pattern Recognition(CVPR), June 1997, Puerto Rico  Laws of Organization in Perceptual Forms, Max Wertheimer, first published as Untersuchungen zur Lehre von der Gestalt II, in Psycologische Forschung, 4, 301-350. Translation published in Ellis, W. (1938). A source book of Gestalt psychology (pp. 71-88). London: Routledge & Kegan Paul. Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues, D. Martin, C. Fowlkes, and J. Malik.   to appear in PAMI  Links Computational Models of Perceptual Organization, talk slides by Stella Yu Laws of Organization in Perceptual Forms, Max Wertheimer The Berkeley Segmentation Dataset and Benchmark Matlab Code for k-means and Expectation Maximization by Carlo Tomasi K-means clustering applet, morphological operators applet,  normalized-cuts mass-spring  movie Presentation by Caglar Ari, Mean Shift: A Robust Approach Toward Feature Space Analysis, Dorin Comaniciu and Peter Meer, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, no.5 , May 2002. (slides)
December 2   Recognition & Image and Video Databases	Recognition Topics Model based and template matching based methods for recognition (slides) Image and Video Databases Topics Retrieval, browsing and other novel applications on large datasests (slides) Links TRECVID - TREC Video Retrieval Evaluation
December 9 Student Presentations	Presentations Avsar Polat Ay, Object Detection using statistics of parts, Henry Schneiderman and Takeo Kanade, International Journal of Computer Vision, (slides) by Hasan Dogu Taskiran, Robust Real-time Object Detection, Paul Viola, Micheal Jones, International Journal on Computer Vision, 2001,  a longer version is submitted to IJCV (slides) by Ayse Kucukyilmaz, Faces and Names in the News Tamara Miller, Alexander C. Berg, Jaety Edwards, Michael Maire, Ryan White, Yee Whye Teh, Eric Learned-Miller, David A. Forsyth, CVPR 2004 (slides) by Cagri Yuzbasioglu, The Earth Mover's Distanceas a Metric for Image Retrieval, Y. Rubner, C. Tomasi, L.J. Guibas, Technical Report STAN-CS-TN-98-86, Computer Science Department, Stanford University, September 1998 (slides) by Berkan Dulek, Recognition by linear combinations of models, S. Ullman and R. Basri, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.13, No.10, October 1991 (slides) by Gonenc Ercan, Learning and Recognizing Human Dynamics in Video Sequences, C. Bregler, CVPR, 1997 (slides) by Kivanc Kose, Feature based methods for structure and motion estimation, Phil Torr and Andrew Zisserman, in Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, R. Szeliski (Eds.), Springer, 2000 (slides)
December 16 Student Presentations	Presentations by Yigithan Dedeoglu, Silhouette-based Human Identification from Body Shape and Gait, R.Collins, R.Gross and J.Shi, Intl' Conference on Face and Gesture, October 2002, pp.351-356 (slides) by Zeynep Erson, Shape Matching and Object Recognition Using Shape Contexts , Serge Belongie, Jitendra Malik and Jan Puzicha, IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), 24(4):509-522, April 2002. (slides) by Onur Oktay, Object Class Recognition by Unsupervised Scale-Invariant Learning, Rob Fergus, Pietro Perona, and Andrew Zisserman, Conference on Computer Vision and Pattern Recognition, (2003). (slides) by Hande Dogan, Body Plans, Forsyth, D.A. and Fleck, M.M., Proc. in Proceedings of Computer Vision and Pattern Recognition (CVPR), 1997 (slides) by Ajda Karabulut, Learning Patterns of Activity Using Real-Time Tracking,  Chris Stauffer, Eric Grimson, , pami, 22(8):747-757, 2000 by Onur Onder, A statistical model for general contextual object recognition,  P. Carbonetto, N. de Freitas, and K. Barnard. . In Eight European Conference on Computer Vision (ECCV), 2004. (slides) by Demir Gokalp, Deformable Shape Detection and Description via Model-Based Region Grouping, S. Sclaroff, and L. Liu, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), 23(5), May, 2001. (slides)
December 23 Student Presentations	Presentations by Sinan Erkustan, Combining Simple Discriminators for Object Discrimination S. Mahamud, M. Hebert, J. Lafferty ECCV 2002. (slides) by Funda Durupinar, Recovering Human Body Configurations: Combining Segmentation and Recognition, G. Mori, X. Ren, A. Efros, and J. Malik, , Computer Vision and Pattern Recognition (to appear), 2004 (slides) by Faysal Basci, Man-Made Structure Detection in Natural Images using a Causal Multiscale Random Field, S. Kumar and M. Hebert,  in proc. IEEE International Conference on Computer Vision and Pattern Recognition (CVPR),  Vol. 1, pp. 119-126,  2003 by Gokhan Akcay, Face Recognition Across Pose and Illumination R. Gross, S. Baker, I. Matthews, and T. Kanade, Handbook of Face Recognition, Stan Z. Li and Anil K. Jain, ed., Springer-Verlag, 2004 (slides) by Ibrahim Demir, Reconstruction of articulated objects from point correspondences in a single uncalibrated image, Camillo J. Taylor, Computer Vision and Image Understanding, Vol. 80 , Issue 3, December 2000. (slides) by Derya Ozkan, Finding and tracking people from the bottom up, Deva Ramanan and David Forsyth,  in Proceedings of Computer Vision and Pattern Recognition (CVPR), Madison, Wisconsin, June 2003 (slides) by Ozgun Babur, Pedestrian detection using wavelet templates, M. Oren and C. Papageorgiou and P. Sinha and E. Osuna, T. Poggio, CVPR, 1997 (slides)
December 30	NO CLASS