ESRGAN Super Resolution Camera Raw De-Bayering

ESRGAN Super Resolution Camera Raw De-Bayering. Performs Camera Raw image de-Bayer using deep learning.

ESRGAN Super Resolution Camera Raw De-Bayer version 1.0.0.

■ Prerequisites ■

Matlab 2022a

Image Processing toolbox

Statistics and Machine Learning toolbox

Deep Learning Toolbox

Parallel Computing Toolbox

■ How to Test ■

Run ESRGAN_Test.m which calls ESRGAN_2xSuperResolution.m

Trained net is loaded on the line 5 of ESRGAN_2xSuperResolution.m

■ How to Perform ESRGAN Super-Resolution to your image file ■

Input Camera RAW image should be RG/GB Bayer pattern layout. Otherwise it is necessary to re-train the network.

Raw image format should be supported by Matlab. It is possible to convert your Camera RAW image to

DNG file format using Adobe DNG Converter (Free of charge) https://helpx.adobe.com/camera-raw/using/adobe-dng-converter.html

Specify your file to filename variable on line 1 of ESRGANRaw_DebayerTest.m and Run it.

After 10 minutes or so, developed.png will be outputted.

■ Limitations ■

・Lens chromatic abberation is not corrected.

・Image brightness may be not correct.

■ How to Train the network ■

Download Flickr2K dataset and place all png files on Flickr2K/Flickr2K_HR.

Run createTrainingSetAll_Flickr2K.m to create Flickr2K_RGB_MatlabF2 folder that contains converted mat files.

Set channel='R' in line 17 and run ESRGANRawRB_Train.m

Set channel='B' in line 17 and run ESRGANRawRB_Train.m

Run ESRGanRawG_Train.m

■ My training result becomes complete white image. How to fix it ■

・Reduce the learning rate.

・Run ESRGAN_Train.m and watch values of lossGenMSE, lossGenFromDisc, lossGenVGG54 on Command Window.

If one value is significantly larger than other two, decrease it.

■ How to get more crisp image ■

Decrease lossGenMSE contribution of ESRGAN_Train.m:399 to get more crisp image. But artifact increases.

■ Changelog ■

Version 1.0.1

・Fixed G-channel offset bug

Version 1.0.0

・Initial release.

■ References ■

Implement Digital Camera Processing Pipeline

https://www.mathworks.com/help/images/end-to-end-implementation-of-digital-camera-processing-pipeline.html

Xintao Wang, et al. ESRGAN: Enhanced super-resolution generative adversarial networks. In ECCVW, 2018.

https://arxiv.org/abs/1809.00219

Ledig, C., Theis, L., Husz ́ar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken,A., Tejani, A., Totz, J., Wang, Z., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR (2017)

https://arxiv.org/pdf/1609.04802.pdf

Single Image Super-Resolution Using Deep Learning

(VDSR is implemented using Matlab Deep Learning Toolbox)

https://www.mathworks.com/help/images/single-image-super-resolution-using-deep-learning.html

Train Generative Adversarial Network (GAN) using Matlab

https://www.mathworks.com/help/deeplearning/ug/train-generative-adversarial-network.html

Monitor GAN Training Progress and Identify Common Failure Modes

https://www.mathworks.com/help/deeplearning/ug/monitor-gan-training-progress-and-identify-common-failure-modes.html

VGG-19 convolutional neural network (Matlab)

https://www.mathworks.com/help/deeplearning/ref/vgg19.html?searchHighlight=VGG19&s_tid=srchtitle