Abstract

Outdoor scene relighting is a challenging problem that requires good understanding of the scene geometry, illumination and albedo. Current techniques are completely supervised, requiring high quality synthetic renderings to train a solution. Such renderings are synthesized using priors learned from limited data. In contrast, we propose a self-supervised approach for relighting. Our approach is trained only on corpora of images collected from the internet without any user-supervision. This virtually endless source of training data allows training a general relighting solution. Our approach first decomposes an image into its albedo, geometry and illumination. A novel relighting is then produced by modifying the illumination parameters. Our solution capture shadow using a dedicated shadow prediction map, and does not rely on accurate geometry estimation. We evaluate our technique subjectively and objectively using a new dataset with ground-truth relighting. Results show the ability of our technique to produce photo-realistic and physically plausible results, that generalizes to unseen scenes.

Paper	Dataset	Supplemental	Code and models

Video Presentation (Short)

Video Presentation (Long)

More Video Results

Bibtex

	@InProceedings{YuSelfRelight20,
			title      = {Self-supervised Outdoor Scene Relighting},
			author     = {Yu, Ye and Meka, Abhimetra and Elgharib, Mohamed and Seidel, Hans-Peter and Theobalt, Christian and Smith, Will},
			booktitle  = {European Conference on Computer Vision (ECCV)},
			year = {2020}
			}