Deco
max planck institut
informatik
mpii logoMinerva of the Max Planck Society
 

Computer Vision for Computer Graphics

Seminar – Summer Semester 2013

Organizers: Christian Theobalt, James Tompkin


Intrinsic Image Decomposition
(Laffont et al. SIGGRAPH ASIA 2012)
Advanced Video Editing
(Bhat et al. EGSR 2007)


Overview


Computer Vision strives to develop algorithms for understanding, interpreting, and reconstructing information about real world scenes from image and video data.

Computer Graphics focuses on image synthesis: on the development of algorithms to build and edit static and dynamic virtual worlds and to display them in photorealistic or stylized ways.

In recent years, these fields have converged more and more since both disciplines create and exploit models describing the visual appearance of objects and scenes. For example, Computer Graphics researchers have started to investigate algorithms to reconstruct detailed models of static and dynamic scenes from image data, such that more believable virtual renderings of real world scenes could be achieved. In a similar manner, Computer Vision research has been able to benefit from the experience of Computer Graphics research to efficiently and effectively model image formation and light transport, which also simplifies scene analysis and reconstruction tasks. Recently, the ever increasing amount of image and video data has further strengthened the links between the two fields, as both communities start to investigate new ways to analyse, structure, and immersively display these data.

In this seminar series, we will cover advanced research topics that cross the boundaries between the fields of Computer Vision and Computer Graphics. These include classic and recent research results that were published at top tier conferences and in top journals. This seminar will cover research papers from the following problem fields:

Target Audience:

The target audience are graduate students in computer science or related fields. Basic knowledge in at least one of the fields of 3D geometry, computer vision, image processing, and computer graphics is required. Every participant will give a talk on a chosen scientific topic. Afterwards, the topic will be discussed within the seminar group. This is a great opportunity for students to improve their research and presentation skills and to learn about the latest developments in computer vision and computer graphics. The seminar language is English.


Photometric Stereo
(Joshi et al. ICCV 2007)
Full-body Performance Capture
(Liu et al. CVPR 2011)


Organization


Contact:

In case you have questions about this seminar, please contact James Tompkin – jtompkin (at) mpi-inf.mpg.de.

Date and Time:

Time: Tuesdays, 16:00 - 18:00 (c.t.)
Room: Building E1.4 (Max-Planck-Institut für Informatik), Room 019
 
April 16th: Presentation of topics. Compulsory for all Participants!
(2013 Slides PPTX (300MB) & PDF without videos (4MB))
April 30th: Classroom lecture: "How to give a good talk"
Slides and Video (90mins - 1.4GB)
May 7th: First seminar talk by participant. All other talks will follow in regular weekly slots.

Registration:

Registration is closed as the seminar series has started.

To register, send an email with your name, matriculation number, current semester and email address to jtompkin (at) mpi-inf.mpg.de. The details of the seminar and the available topics will be discussed in the first class on April 16. Attendance is required in all classes. Please note that the number of available presentation slots is limited to 12. We will try to accommodate all requests for participation but in case we have more requests than available slots, we will assign slots on a first come first serve basis.

Mailing List:

All official announcements will be made through the mailing list. Please subscribe to it.

itvc@lists.mpi-inf.mpg.de


Body Reshaping in Video
(Jain et al. SIGGRAPH ASIA 2010)
Stereo Facial Performance Capture
(Valgaerts et al. SIGGRAPH ASIA 2012)


Structure (Summary)


Each week:

After seminar series:


Structure (Detail)


Part 1: Presentation (40% of mark)

Each participant will perform a detailed study of one research topic by means of two representative scientific papers. The participant will present the main ideas of the papers in a presentation of approx. 45 minutes. Usually, this means that content has to be selected from the two representative papers, and it will often not be possible to discuss all results presented therein. Instead, try to focus on the topic and find a common thread linking the two papers on which you can build your presentation. Work on the topic usually requires reading and understanding of the papers, as well as acquiring background information necessary to understand the topic. For extra background knowledge, some of the papers and books referenced in the papers may have to be consulted.

At the beginning of your presentation, you will spend 5 minutes summarizing the previous week's presentations. The aim of this section is to recap for the audience, and for you to tease out connections between the content across weeks. All of the topics are connected in one way or another, and in these 5 minutes you should try to establish these connections as a way to part-motivate your presentation.

If you use formulas, make sure that all symbols are introduced properly. Similarly, make sure that figures are labelled correctly and that new terminology is introduced appropriately. You may assume that your audience is familiar with topics that have been presented in class earlier, and have read the papers for your week. It is very important for a good presentation to find a balance between overview and detail. Most importantly, the general theme of your topic should become clear from your presentation. Your goal is that the other participants will have gained a general understanding of your topic after your presentation. You can pick one or two things that you find interesting and present those in more detail. Finally, prepare your presentation on time and plan to spend some time on practising your talk. Our feedback afterwards will help you improve future presentations.

Part 2: Discussion (20% of mark across weeks)

The presentation will be followed by a discussion among the seminar participants. This discussion will be chaired by the discussion leader, who will be one of the participants. Before the presentation, AT RANDOM, one participant will be chosen to be the discussion leader. Their job is to direct discussion such that the strengths and weaknesses of the techniques and of the field are discussed, that open questions and future directions of the work are discussed, and that questions of other participants are fielded.

Each week, every participant will submit 2+ questions that they would like answered on the upcoming week's papers. These questions will be submitted 1 day before the seminar. Before the presentation, these questions will be given to the discussion leader, and it is their job to integrate them into the discussion. For participants, these questions act as evidence of participation, and your overall participation score will be based on the quality of your questions and your involvement in the discussion.

Participants are expected to read every paper in preparation for the upcoming presentations. We expect students to actively engage in discussions for further understanding of the presented material. We aim at a creative atmosphere - ideas developed during the seminar work might lead to Master thesis projects.

Part 3: Written Report (40% of mark)

In addition to the in-class presentation, we require a written report on the chosen topic. This report should summarise the main ideas of the accompanying papers and discuss limitations and drawbacks of the works. In addition, participants should develop and sketch their own idea of how to address one specific shortcoming. The report should consist of 6-8 pages covering the presented papers and about 2-3 pages covering the improvement proposed by the student.

The written report uses a template of a major computer vision conference, the IEEE Conference on Computer Vision and Pattern Recognition. Please use the review version of this template, as this makes it easier to refer to certain parts of your text. For your convenience we provide the template links here: .tar.gz (Linux) or .zip (Windows). The format of the written report has to be PDF. Please keep this in mind when using the Word template.

We will grade your report based on a number of criteria. As for the presentation, focus on getting the "big picture" and present your topic in context. Try to link the two articles in a coherent text - a structure with one paper following the other is usually insufficient. Also, try to avoid copying the structure of the original papers, do not stick too closely to the original text, and use your imagination to find a way of presenting the topic in your own words. Equations can of course be reproduced. The articles have been chosen such that they provide different views on a common theme. Try to exploit this - it is an opportunity to demonstrate your understanding of a difficult topic – how you can find connections to reduce concepts and look through the ‘academic apparatus’ of the writing.

Keep in mind to structure your paper well in a formal sense: An abstract provides a short overview. The introduction expands on that, explains why the problem is unsolved/hard, and motivates the use of the technique. In a technical section you present your topic and the details that are of interest to you. Here you can also expand on the subject, i.e. add your own ideas. Finally, a conclusion summarises the paper. This is also the place where you can add your own opinions about the topic.

We encourage you to use additional literature in the development of your own ideas. We expect at least 3-4 additional references, if you want to include more, you are welcome to do so. Cite the references appropriately, e.g. if you use concepts from different publication or if you want to link to additional information. Use these other references to create your ‘big picture’ presentation of the field.

Your own ideas can be developed from scratch - if you have a great idea, try to sketch it and a possible solution. The report is also an opportunity to take some of the discussion from the group and integrate and explore the gained context and ideas. You can also discuss limitations or ideas that you feel are missing from the papers. A further possibility is to research recent developments following the papers of your topic. If you pursue this route, you could base your "own ideas" section on several follow-up publications, but make sure that you do not simply copy parts of other papers! A good resource for free links to research articles is Google Scholar. The "cited by" option often links to useful related work. Another source of freely available scientific articles is Citeseer. If you can only find a pay link, send us an email - we will try to get you a copy.

Your paper should be emailed to us by 30th August 2013.


How to Read an Academic Paper


Just as you wouldn't read a website or newspaper in the same way that you would read a novel, there are both efficient and inefficient ways to read academic papers. Learning to read a new kind of material is hard work, and you should try to adapt your reading style to accommodate the characteristics of the medium. It might take some time to find what reading style or method works best for the individual, but some general guidelines are broadly applicable. We present a list of online references from a computer science perspective which may help you adjust more quickly to the reading task at hand:




How to Write an Academic Paper


Academic writing is similarly different from other forms of writing. We present a list of online references which may help you write your reports:




How to Give an Academic Presentation


Christian discussed this topic at length in his "How to give a good talk" lecture, but here's more references which you might find useful:




List of Papers


All discussion questions per week: HERE.


Topic Paper Title Authors Published at Contact Presenter Date Seminar Slides

Image-based:
Illusion of motion

Exploring Photobios

Eulerian Video Magnification

Ira Kemelmacher-Shlizerman, Eli Shechtman, Rahul Garg, Steven M. Seitz

Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand, William T. Freeman

SIGGRAPH 2011

SIGGRAPH 2012

James Tompkin
Office hours: Email for appointment.

Amir H. Moin

May 7th

PDF & ODP & Paper Videos & Seminar Video

Image-based:
Human Composition

Recognizing Action at a Distance

Photo Clip Art

A. A. Efros, A. C. Berg, G. Mori and J. Malik

J.-F. Lalonde, D. Hoiem, A. A. Efros, C. Rother, J. Winn, A. Criminisi

ICCV 2003

SIGGRAPH 2007

Kwang In Kim

Amirhossein Kardoost

May 21st

PPTX & Paper Videos & Seminar Video

Image-based:
Patch correspondence

PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing

NRDC: Non-Rigid Dense Correspondence with Applications for Image Enhancement

Connelly Barnes, Eli Shechtman, Adam Finkelstein, Dan B Goldman

Yoav HaCohen, Eli Shechtman, Dan B. Goldman, Dani Lischinski

SIGGRAPH 2009

SIGGRAPH 2011

James Tompkin
Office hours: Email for appointment.

Natali Dedik

May 28th

PPT & PDF & Seminar Video

Shape capture:
Visual Hull Foundations

The Visual Hull Concept for Silhouette-Based Image Understanding

Visual Hull Alignment and Refinement Across Time: A 3D Reconstruction Algorithm Combining Shape-From-Silhouette with Stereo

A. Laurentini

German K.M. Cheung, Simon Baker, Takeo Kanade

PAMI 1994

CVPR 2003

Ahmed Elhayek

* Reduced - short supervisor presentation *

June 4th

PPTX & Pooh Video & Seminar Video

Pose estimation:
Foundations

Tracking People with Twists and Exponential Maps

Optimization and Filtering for Human Motion Capture - A Multi-layer Framework

Bregler and Malik

Juergen Gall, B. Rosenhahn, Thomas Brox, and Hans-Peter Seidel

CVPR 1998

IJCV 2008

Thomas Helten

* Reduced - short supervisor presentation *

June 11th

PPTX & Seminar Video

Shape capture:
Performance capture

Performance Capture from Sparse Multi-view Video

Motion Capture Using Joint Skeleton Tracking and Surface Estimation

Edilson de Aguiar, Carsten Stoll, Christian Theobalt, Naveed Ahmed, Hans-Peter Seidel, Sebastian Thrun

Juergen Gall, Carsten Stoll, Edilson de Aguiar, Christian Theobalt, Bodo Rosenhahn, and Hans-Peter Seidel

SIGGRAPH 2008

CVPR 2009

Christian Theobalt
Office hours: Email for appointment.

Yeara Kozlov

June 18th

PDF Slides & Seminar Video

Shape capture:
Facial performance capture

High-Quality Passive Facial Performance Capture using Anchor Frames

Lightweight Binocular Facial Performance Capture under Uncontrolled Lighting

T. Beeler, F. Hahn, D. Bradley, B. Bickel, P. Beardsley, C. Gotsman, M. Gross

Levi Valgaerts, Chenglei Wu, Andrés Bruhn, Hans-Peter Seidel, Christian Theobalt

SIGGRAPH 2011

SIGGRAPH Asia 2012

Pablo Garrido

Darya Dedik

June 25th

PDF Slides & Seminar Video

Pose estimation:
Alternatives

Fast Articulated Motion Tracking using a Sums of Gaussians Body Model

Real-Time Human Pose Recognition in Parts from a Single Depth Image

Carsten Stoll, Nils Hasler, Juergen Gall, Hans-Peter Seidel, Christian Theobalt

Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby Sharp, Mark Finocchio, Richard Moore, Alex Kipman, and Andrew Blake

ICCV 2011

CVPR 2011

Srinath Sridhar

Zornitsa Kostadinova

July 2nd

PDF Slides & Seminar Video

Illumination:
Shape and Reflectance

Shape from Varying Illumination and Viewpoint

Shading-based Dynamic Shape Refinement from Multi-view Video under General Illumination

Neel Joshi, David Kriegman

Chenglei Wu, Kiran Varanasi, Yebin Liu, Hans-Peter Seidel, Christian Theobalt

ICCV 2007

ICCV 2011

Chenglei Wu

* Reduced - short supervisor presentation *

July 9th

PPTX Slides, Paper Video 1, Paper Video 2, Seminar Video

Illumination:
Decomposition

Webcam Clip Art: Appearance and Illuminant Transfer from Time-lapse Sequences

Coherent Intrinsic Images from Photo Collections

Jean-François Lalonde, Alexei A. Efros, Srinivasa G. Narasimhan

Pierre-Yves Laffont, Adrien Bousseau, Sylvain Paris, Frédo Durand, George Drettakis

SIGGRAPH Asia 2009

SIGGRAPH Asia 2012

Chenglei Wu

* Reduced - short supervisor presentation *

July 16th

PPTX Slides, Paper Video 1, Paper Video 2, Seminar Video

Previous 'possible' topics that are now excluded from the schedule. Here for reference if you want to explore more of the field.

Topic Paper Title Authors Published at Contact Presenter Date Seminar Slides

Image-based:
Super resolution

Bayesian Image Super-resolution

Bayesian Methods for Image Super-resolution

Michael E. Tipping and Christopher M. Bishop

L. C. Pickup, D. P. Capel, S. J. Roberts, A. Zisserman

NIPS 2002

The Computer Journal 2007

N/A

N/A

N/A

N/A

Shape capture:
Multi-view Stereo

Multi-View Stereo Revisited

Joint Estimation of Motion, Structure and Geometry from Stereo Sequences

Michael Goesele, Brian Curless, Steven M. Seitz

Levi Valgaerts, Andrés Bruhn, Henning Zimmer, Joachim Weickert, Carsten Stoll, Christian Theobalt

CVPR 2006

ECCV 2010

N/A

N/A

N/A

N/A

Shape capture:
Facial performance capture 2

Face Transfer with Multilinear Models

Video Face Replacement

Daniel Vlasic; M. Brand; Hans-Peter Pfister; Jovic Popovic

Kevin Dale; Kalyan Sunkavalli; Micah K. Johnson; Daniel Vlasic; Wojciech Matusik; Hanspeter Pfister

SIGGRAPH 2005

SIGGRAPH Asia 2011

N/A

N/A

N/A

N/A

Pose estimation:
Hands

Capturing Natural Hand Articulation

Motion Capture of Hands in Action using Discriminative Salient Points

Ying Wu, John Y. Lin, Thomas S. Huang

Luca Ballan, Aparna Taneja, Jürgen Gall, Luc Van Gool, Marc Pollefeys

ICCV 2001

ECCV 2012

N/A

N/A

N/A

N/A

Data-driven Dynamics:
Models

SCAPE: Shape completion and animation of people

DRAPE: DRessing Any PErson

D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, J. Davis

P. Guan, L. Reiss, D. Hirshberg, A. Weiss and M.J. Black

SIGGRAPH 2005

SIGGRAPH 2012

N/A

N/A

N/A

N/A

Data-driven Dynamics:
Skin Deformation

Capturing and Animating Skin Deformation in Human Motion

Capture and statistical modeling of arm-muscle deformations

Sang Il Park, Jessica K. Hodgins

T. Neumann, K. Varanasi, N. Hasler, M. Wacker, M. Magnor, C. Theobalt

SIGGRAPH 2006

EG 2013

N/A

N/A

N/A

N/A

Data-driven Dynamics:
Hidden spaces

Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces

Trajectory Space: A Dual Representation for Nonrigid Structure from Motion

A. Safonova, Jessica Hodgins, and Nancy Pollard

I. Akhter, Y. Sheikh, S. Khan, T. Kanade

SIGGRAPH 2004

PAMI 2010

N/A

N/A

N/A

N/A

Data-driven Dynamics:
Hidden spaces/Cloth

Stable spaces for real-time clothing

Bilinear spatiotemporal basis models

E. Aguilar, L. Sigal, A. Treuille, Jessica Hodgins

I. Akhter, T. Simon, S. Khan, I. Matthews, Y. Sheikh

SIGGRAPH 2010

SIGGRAPH 2012

N/A

N/A

N/A

N/A

Applications

Using Photographs to Enhance Videos of a Static Scene

MovieReshape: Tracking and Reshaping of Humans in Videos

Pravin Bhat, C. Lawrence Zitnick, Noah Snavely, Aseem Agarwala, Maneesh Agrawala, Brian Curless, Michael Cohen, Sing Bing Kang

Arjun Jain, Thorsten Thormählen, Hans-Peter Seidel, Christian Theobalt

EGSR 2007

SIGGRAPH Asia 2010

N/A

N/A

N/A

N/A




Search MPII (type ? for help)
, (paper)