IEEE Transactions on Pattern Analysis and Machine Intelligence
Efficient Learning-based Image Enhancement

Younghee Kwon Kwang In Kim James Tompkin Jin Hyung Kim Christian Theobalt
MPI für Informatik KAIST Lancaster University Harvard University SEAS

Top: Bicubic resampling. Middle: Kim and Kwon [6]. Bottom: Our method with fast-learning of a dataset-specific super-resolution.
Left: Face-specific super-resolution: [6] △PSNR: 1:18dB, △SSIM: 0:032; our method △PSNR: 2:10dB, △SSIM: 0:054.
Right: Document-specific super-resolution; 2x and 3x magnification.

Improving the quality of degraded images is a key problem in image processing, but the breadth of the problem leads to domain-specific approaches for tasks such as super-resolution and compression artifact removal. Recent approaches have shown that a general approach is possible by learning application-specific models from examples; however, learning models sophisticated enough to generate high-quality images is computationally expensive, and so specific per-application or per-dataset models are impractical. To solve this problem, we present an efficient semi-local approximation scheme to large-scale Gaussian processes. This allows efficient learning of task-specific image enhancements from example images without reducing quality. As such, our algorithm can be easily customized to specific applications and datasets, and we show the efficiency and effectiveness of our approach across five domains: single-image super-resolution for scene, human face, and text images, and artifact removal in JPEG- and JPEG 2000-encoded images.

author = {Younghee Kwon and Kwang In Kim and James Tompkin and Jin Hyung Kim and Christian Theobalt},
title = {Efficient Learning of Image Super-resolution and Compression Artifact Removal
with Semi-local {Gaussian} Processes},
booktitle = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {37},
number = {9},
pages = {1792--1805},
year = {2015}

PDF (3 MB)
  Supplemental Material
PDF (32 MB)

Super-resolution Comparison Results

JPEG Artifact Removal Comparison Results

JPEG2000 Artifact Removal Comparison Results

We thank the anonymous TPAMI reviewers for their feedback. The ideas presented have greatly profited from discussions with Christian Walder, Nils Hasler, Miguel Granados, and Carsten Stoll. Part of this work was completed while Y. Kwon was with KAIST, while K. I. Kim was with the machine learning group, Saarland University, and while K. I. Kim and J. Tompkin were with the GVV group, Max-Planck-Institute for Informatics. J. Tompkin acknowledges NSF CGV-1110955.
Images courtesy of The Facial Recognition Technology (FERET) Database, Kodak Lossless True Color Image Suite, The Berkeley Segmentation Dataset, Lena courtesy of Playboy, and Fabio courtesy of Fabio Lanzoni (agent: Eric Ashenberg) via Deanna Needell - thanks!