Bilkent University
Department of Computer Engineering
S E M I N A R
Multi-Task Networks for Coloring, Counting and Cliff-Diving
Prof. Dr. Cees Snoek
University of AmsterdamIn this talk we present recent progress on colorizing gray-scale photographs, counting arbitrary objects and detecting human activities in video. While each of these computer vision challenges have a different purpose, they all benefit from a multi-task formulation. For coloring we start from the observation that human beings perceive and distinguish colors based on the semantic categories of objects, while image colorization algorithms still suffer from limited semantic understanding. To address this shortcoming, we propose to exploit pixelated semantics to guide the image colorization. For counting, the leading approaches start from point annotations per object from which they construct density maps. Then, their training objective transforms input images to density maps. We introduce ways to repurpose the points for free with a segmentation branch to focus on areas of interest and a context branch to regularize the overall density estimation. For cliff-diving and other human activities of interest, the two-stream detection network based on RGB and flow provides state-of-the-art accuracy at the expense of a large model-size and heavy computation. We propose to embed RGB and optical-flow into a single two-in-one stream network with new motion condition and RGB modulation layers, leading to better accuracy and faster run-time. We will illustrate the capabilities of our colorization, counting and action detection networks with qualitative and quantitative experiments on several image and video collections.
Bio: Cees Snoek is a full professor in artificial intelligence at the University of Amsterdam, where he heads the Intelligent Sensory Information Systems Lab. He is also a director of the QUVA Lab, a joint research lab on deep learning and computer vision with Qualcomm, and the AIM Lab, a joint research lab on medical imaging with the Inception Institute of Artificial Intelligence. At University spin-off Kepler Vision Technologies he acts as Chief Scientific Officer. Professor Snoek is also the director of the master program in Artificial Intelligence and co-founder of the Innovation Center for Artificial Intelligence. He received the M.Sc. degree in business information systems (2000) and the Ph.D. degree in computer science (2005) both from the University of Amsterdam, The Netherlands. He was previously an assistant (2011-2013) and associate professor (2013-2017) at the University of Amsterdam, as well as Visiting Scientist at Carnegie Mellon University (2003), Fulbright Junior Scholar at UC Berkeley (2010-2011), head of R&D at University spin-off Euvision Technologies (2011-2014) and managing principal engineer at Qualcomm Research Europe (2014-2017). His research interests focus on video and image recognition.
DATE: 07 November 2019, Thursday @ 13:00
PLACE: EA 409