We will extend the augmented reality framework for crowd simulation. Possible extensions are related to seamless integration of real and synthetic objects in an augmented reality crowd simulation scenario, design and implementation of algorithms for camera registration and tracking, extending the communication models for information exchange between the individual agents in a crowd, and simulating the effects of information exchange in different crowd simulation scenarios.

Our project Turkish Sign Language (TID) converter recognizes hand postures and gestures that are used in Turkish Sign Language and by matching them with its existing database of signs finds its correspondent letters and words in the verbal language and outputs those in text and sound. The users may also add new signs to the system for their personal use, which will be converted to the letters or words they desire in order to extend usage of program to more people.

This software will be extended so that it will recognize a larger vocabulary, text-to-speech conversion, and other possible extensions.

Extraction of 3D models from satellite imagery and digital photos is one of the hot topics nowadays, which is already supported by some of the commmercial applications. The project will add some new functionalities to an existing system for 3D city model generation. Currently, the system is capable of generating city models manually by adding new buildings with different properties (windows, doors, balconies, number of floors, etc.) The system also capable of generating city models automatically by adding buildings with a random number of floors, windows, balconies, etc. Possible extensions would be buildings other than rectangular floor plans, different types of windows, doors, balconies, developing efficient algorithms for extracting floor plans from satellite (LIDAR) images so that generating city models for larger cities is possible.

Producing 3D human models that show emotions through postures and facial expressions. The models need to be exported to Cal3d format and different animations, such as walking, sitting etc. should support emotional expressions. e.g., angry walking, sad walking, happy running, etc. (see pictures below). Necessary information for posture creation can be found in psychology literature [Walbott, H.G., Bodily Expression of Emotion, European Journal of Social Psychology, 28, 879-896 (1998), Works of Paul Ekman]. A program that automates model creation with bodily and facial expression of emotions will be implemented within the scope of the project.

Implementing efficient and detailed kinematic walking for articulated 3D human models for crowd synchronization. The program should enable the creation of multiple human models. Also, a simple pyhsically-based walking model could be implemented within the scope of the project. Recent ACM SIGGRAPH papers (e.g., [SIMBICON: Simple Biped Locomotion Control Yin, K., Loken, K., and van de Panne, M., ACM Transactions on Graphics 26(3),2007]) could be implemented for this purpose.

Implementing a program that converts between ASCII-based building files and .obj files. Building files are in .txt format and they provide information about the position and orientation of rooms, walls and gates. This format can be easily adapted to outdoor environments as walls can be mapped to the walls of buildings, rooms can be mapped to pedestrian walkways, parks and pavements and gates can be mapped to the locations of traffic lights where pedestrians can cross the road or entrances of buildings. Currently, this process is performed manually; a system that automates this process would be very useful. As an extension of this project, we need city and indoors models with textures as .obj files. Producing these models with Maya or 3DSMax could be part of this senior project.

Implementing Interactive procedural street modeling ("Interactive procedural street modeling", Guoning Chen, Gregory Esch, Peter Wonka, Pascal Müller, Eugene Zhang, ACM Transactions on Graphics, (Proc. of ACM SIGGRAPH 2008)) for crowd simulation also can be considered as an extension of this project.

Implementing a program that adds Style and Emotion to Motion Captured Data. This program should enable the user to modify captured human behaviors using a set of style and mood parameters. Example Papers: [Eugene Hsu, Kari Pulli, Jovan Popovic, Style translation for human motion, ACM Transactions on Graphics (TOG), Vol.24, No.3, July 2005], [Ari Shapiro, Yong Cao , Petros Faloutsos, Style components, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada], You may also add anticipation and follow-through effects: [Jue Wang , Steven M. Drucker , Maneesh Agrawala , Michael F. Cohen, The cartoon animation filter, ACM Transactions on Graphics (TOG), Vol.25, No.3, July 2006] and for a brief introduction; [John Lasseter, Principles of traditional animation applied to 3D computer animation, Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, pp.35-44, August 1987].

Makadia et al. [1] describes a propagation method in which a test image is annotated by propagating the tags from similar training images. Given a test image, first similar images which are already annotated are retrieved using a similarity measure. This is the retrieval phase. After similar annotated images are retrieved the annotation data is used or propagated to the test image. Although the training set should be large enough to find reasonable neighbors for a succesful propagation, this approach has shown to provide successful results. In [2], we describe a procedure based on constructing codebooks for each semantic label. In the training phase for each semantic/annotation label you retrieve all images that are tagged with it. Then by using a descriptor, e.g. SIFT, you construct a codebook. In the testing phase of the method, given a test image, for each extracted feature you calculate the distance between it and the nearest visual word of a codebook. The summation of distances then gives you the image distance to that particular codebook. In the end you use k semantics of codebooks having the lowest distance with the test image. Improvements can be made in areas of:

• Integrating contextual information [3].
• Integrating spatial information of the visual words if codebook framework is going to be used [2]
• Weighting the visual words according to the discriminative power.
• Combining other feature descriptors to your framework.

References

1. Ameesh Makadia, Vladimir Pavlovic, and Sanjiv Kumar (2010). Baselines for Image Annotation. In International Journal of Computer Vision.
2. Fatih Cakir (2011). Nearest Neighbor based Metric Functions for Indoor Scene Recognition. MS thesis. Bilkent University.
3. Yu Xiang; Xiangdong Zhou; Zuotao Liu; Tat-Seng Chua; Chong-Wah Ngo; , "Semantic context modeling with maximal margin Conditional Random Fields for automatic image annotation," Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on , vol., no., pp.3368-3375, 13-18 June 2010.

You will design and implement an MPEG-7 compatible image search engine, similar to Google images. The system will consist of 3 main components:

1. Image Indexing: Processing images (segmenting images into regions; extracting low-level color, texture, shape features; annotating the images and regions with keywords, e.g., car, tree, animal, sea, sky, face,...) and producing an MPEG-7 compatible XML file, which will be stored in an XML database.
2. Query Interface: A graphical user interface which will enable users to formulate keyword and similarity-based image queries on the image database and view/manipulate the search results.
3. Query Processing: Receiving user queries, executing them and returning the most relevant image list to the clients.

In this project, you will develop an MPEG-7 compatible audio indexing and retrieval system (BilAudio-7). You will be building upon a partially completed version of BilAudio-7. The system will consist of 3 main components:

1. Audio Indexing: Segmenting/annotating audio data and producing an MPEG-7 compatible XML file, which will be stored in an XML database.
2. Query Interface: A graphical user interface which will enable users to formulate keyword and similarity-based audio queries on the audio database and view/manipulate the search results.
3. Query Processing: Receiving user queries, executing them and returning the most relevant list of audio segments to the clients.

MPEG-7 is an ISO standard developed by MPEG group to standardize multimedia indexing and retrieval and make multimedia data (audio/video/image) as searchable as text. It is necessary to process the multimedia data and extract low-level/high-level features for indexing. In this project, you will develop an MPEG-7 annotation tool to annotate audio, video and images. The tool should have an easy-to use GUI and facilities to minimize the amount of human effort and time for annotation.

1. C++ code is available for the extraction of low-level MPEG-7 features
2. The system will be developed in C++ and should be able to run both on Linux and Windows.