update

README.md

@@ -1 +1,65 @@
-# SelfDeblur
+## Neural Blind Deconvolution Using Deep Priors 
+[[arxiv]()] [[supp](supplementary/SelfDeblur_supp.pdf)]
+
+### Introduction
+Blind deconvolution is a classical yet challenging low-level vision problem with many real-world applications.
+Traditional maximum a posterior (MAP) based methods rely heavily on fixed and handcrafted priors that certainly are insufficient in characterizing clean images and blur kernels, and usually adopt specially designed alternating minimization to avoid trivial solution.
+In contrast, existing deep motion deblurring networks learn from massive training images the mapping to clean image or blur kernel, but are limited in handling various complex and large size blur kernels.
+To connect MAP and deep models, we in this paper present two generative networks for respectively modeling the deep priors of clean image and blur kernel, and propose an unconstrained neural optimization solution to blind deconvolution (SelfDeblur).
+Experimental results show that our SelfDeblur can achieve notable quantitative gains as well as more visually plausible deblurring results in comparison to state-of-the-art blind deconvolution methods on benchmark datasets and real-world blurry images.
+
+
+## Prerequisites
+- Python 3.6, PyTorch >= 1.0 
+- Requirements: opencv-python, tqdm
+- Platforms: Ubuntu 16.04, cuda-10.0 & cuDNN v-7.5
+- MATLAB for computing [evaluation metrics](statistic/)
+
+
+## Datasets
+
+SelfDeblur is evaluated on datasets of Levin et al. [1] and Lai et al. [2]. 
+Please download the testing datasets from [BaiduYun]()
+or [OneDrive](), 
+and place the unzipped folders into `./datasets/`.
+
+
+## Getting Started
+
+### 1) Run SelfDeblur
+
+We have placed our learned deep models to [BaiduYun]() and [OneDrive](). Please download these models and place the unzipped folders into `./results/`. 
+
+Run shell scripts to deblur:
+```bash
+bash demo_levin.sh 
+bash demo_lai.sh   
+```
+All the deblurring results are also available at [BaiduYun]() and [OneDrive]().
+You can place the downloaded results into `./results/`, and directly compute all the [evaluation metrics](statistic/) in this paper.  
+
+### 2) Evaluation metrics
+
+We also provide the MATLAB scripts to compute the average PSNR and SSIM values reported in the paper.
+
+
+```Matlab
+ cd ./statistic
+ run statistic_Levin.m
+ run statistic_Lai.m
+```
+
+
+SelfDeblur succeeds in simultaneously estimating blur kernel and generating clean image. 
+<img src="results/demo/levin.png" width="767px"/>
+<img src="results/demo/lai.jpg" width="767px"/> 
+
+
+## References
+[1] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman. Understanding and evaluating blind deconvolution algorithms. In IEEE CVPR 2009. 
+
+[2] W.-S. Lai, J.-B. Huang, Z. Hu, N. Ahuja, and M.-H. Yang. A comparative study for single image blind deblurring. In IEEE CVPR 2016.
+
+
+
+

statistic/psnr.m

@@ -0,0 +1,29 @@
+function [ PSNR ] = psnr( f1,f2 )
+%PSNR Summary of this function goes here
+%   Detailed explanation goes here
+
+
+% % MSE = E( (img£­Eimg)^2 ) 
+% %     = SUM((img-Eimg)^2)/(M*N);
+% function ERMS = erms(f1, f2)
+% %
+% e = double(f1) - double(f2);
+% [m, n] = size(e);
+% ERMS = sqrt(sum(e.^2)/(m*n));
+% % PSNR=10log10(M*N/MSE); 
+% function PSNR = psnr(f1, f2)
+%
+k=1;
+if max(f1(:))>2
+k = 8;
+end
+
+%k
+fmax = 2.^k - 1;
+a = fmax.^2;
+e = double(f1) - double(f2);
+[m, n] = size(e);
+MSE=sum(sum(e.^2))/(m*n);
+PSNR = 10*log10(a/MSE);
+end
+

statistic/ssim.m

@@ -0,0 +1,213 @@
+function [mssim, ssim_map] = ssim(img1, img2, K, window, L)
+
+% ========================================================================
+% SSIM Index with automatic downsampling, Version 1.0
+% Copyright(c) 2009 Zhou Wang
+% All Rights Reserved.
+%
+% ----------------------------------------------------------------------
+% Permission to use, copy, or modify this software and its documentation
+% for educational and research purposes only and without fee is hereby
+% granted, provided that this copyright notice and the original authors'
+% names appear on all copies and supporting documentation. This program
+% shall not be used, rewritten, or adapted as the basis of a commercial
+% software or hardware product without first obtaining permission of the
+% authors. The authors make no representations about the suitability of
+% this software for any purpose. It is provided "as is" without express
+% or implied warranty.
+%----------------------------------------------------------------------
+%
+% This is an implementation of the algorithm for calculating the
+% Structural SIMilarity (SSIM) index between two images
+%
+% Please refer to the following paper and the website with suggested usage
+%
+% Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image
+% quality assessment: From error visibility to structural similarity,"
+% IEEE Transactios on Image Processing, vol. 13, no. 4, pp. 600-612,
+% Apr. 2004.
+%
+% http://www.ece.uwaterloo.ca/~z70wang/research/ssim/
+%
+% Note: This program is different from ssim_index.m, where no automatic
+% downsampling is performed. (downsampling was done in the above paper
+% and was described as suggested usage in the above website.)
+%
+% Kindly report any suggestions or corrections to [email protected]
+%
+%----------------------------------------------------------------------
+%
+%Input : (1) img1: the first image being compared
+%        (2) img2: the second image being compared
+%        (3) K: constants in the SSIM index formula (see the above
+%            reference). defualt value: K = [0.01 0.03]
+%        (4) window: local window for statistics (see the above
+%            reference). default widnow is Gaussian given by
+%            window = fspecial('gaussian', 11, 1.5);
+%        (5) L: dynamic range of the images. default: L = 255
+%
+%Output: (1) mssim: the mean SSIM index value between 2 images.
+%            If one of the images being compared is regarded as 
+%            perfect quality, then mssim can be considered as the
+%            quality measure of the other image.
+%            If img1 = img2, then mssim = 1.
+%        (2) ssim_map: the SSIM index map of the test image. The map
+%            has a smaller size than the input images. The actual size
+%            depends on the window size and the downsampling factor.
+%
+%Basic Usage:
+%   Given 2 test images img1 and img2, whose dynamic range is 0-255
+%
+%   [mssim, ssim_map] = ssim(img1, img2);
+%
+%Advanced Usage:
+%   User defined parameters. For example
+%
+%   K = [0.05 0.05];
+%   window = ones(8);
+%   L = 100;
+%   [mssim, ssim_map] = ssim(img1, img2, K, window, L);
+%
+%Visualize the results:
+%
+%   mssim                        %Gives the mssim value
+%   imshow(max(0, ssim_map).^4)  %Shows the SSIM index map
+%========================================================================
+
+
+if (nargin < 2 || nargin > 5)
+   mssim = -Inf;
+   ssim_map = -Inf;
+   return;
+end
+
+if (size(img1) ~= size(img2))
+   mssim = -Inf;
+   ssim_map = -Inf;
+   return;
+end
+
+[M N] = size(img1);
+
+if (nargin == 2)
+   if ((M < 11) || (N < 11))
+	   mssim = -Inf;
+	   ssim_map = -Inf;
+      return
+   end
+   window = fspecial('gaussian', 11, 1.5);	%
+   K(1) = 0.01;					% default settings
+   K(2) = 0.03;					%
+   L = 255;                                     %
+end
+
+if (nargin == 3)
+   if ((M < 11) || (N < 11))
+	   mssim = -Inf;
+	   ssim_map = -Inf;
+      return
+   end
+   window = fspecial('gaussian', 11, 1.5);
+   L = 255;
+   if (length(K) == 2)
+      if (K(1) < 0 || K(2) < 0)
+		   mssim = -Inf;
+   		ssim_map = -Inf;
+	   	return;
+      end
+   else
+	   mssim = -Inf;
+   	ssim_map = -Inf;
+	   return;
+   end
+end
+
+if (nargin == 4)
+   [H W] = size(window);
+   if ((H*W) < 4 || (H > M) || (W > N))
+	   mssim = -Inf;
+	   ssim_map = -Inf;
+      return
+   end
+   L = 255;
+   if (length(K) == 2)
+      if (K(1) < 0 || K(2) < 0)
+		   mssim = -Inf;
+   		ssim_map = -Inf;
+	   	return;
+      end
+   else
+	   mssim = -Inf;
+   	ssim_map = -Inf;
+	   return;
+   end
+end
+
+if (nargin == 5)
+   [H W] = size(window);
+   if ((H*W) < 4 || (H > M) || (W > N))
+	   mssim = -Inf;
+	   ssim_map = -Inf;
+      return
+   end
+   if (length(K) == 2)
+      if (K(1) < 0 || K(2) < 0)
+		   mssim = -Inf;
+   		ssim_map = -Inf;
+	   	return;
+      end
+   else
+	   mssim = -Inf;
+   	ssim_map = -Inf;
+	   return;
+   end
+end
+
+
+img1 = double(img1);
+img2 = double(img2);
+
+% automatic downsampling
+f = max(1,round(min(M,N)/256));
+%downsampling by f
+%use a simple low-pass filter 
+if(f>1)
+    lpf = ones(f,f);
+    lpf = lpf/sum(lpf(:));
+    img1 = imfilter(img1,lpf,'symmetric','same');
+    img2 = imfilter(img2,lpf,'symmetric','same');
+
+    img1 = img1(1:f:end,1:f:end);
+    img2 = img2(1:f:end,1:f:end);
+end
+
+C1 = (K(1)*L)^2;
+C2 = (K(2)*L)^2;
+window = window/sum(sum(window));
+
+mu1   = filter2(window, img1, 'valid');
+mu2   = filter2(window, img2, 'valid');
+mu1_sq = mu1.*mu1;
+mu2_sq = mu2.*mu2;
+mu1_mu2 = mu1.*mu2;
+sigma1_sq = filter2(window, img1.*img1, 'valid') - mu1_sq;
+sigma2_sq = filter2(window, img2.*img2, 'valid') - mu2_sq;
+sigma12 = filter2(window, img1.*img2, 'valid') - mu1_mu2;
+
+if (C1 > 0 && C2 > 0)
+   ssim_map = ((2*mu1_mu2 + C1).*(2*sigma12 + C2))./((mu1_sq + mu2_sq + C1).*(sigma1_sq + sigma2_sq + C2));
+else
+   numerator1 = 2*mu1_mu2 + C1;
+   numerator2 = 2*sigma12 + C2;
+	denominator1 = mu1_sq + mu2_sq + C1;
+   denominator2 = sigma1_sq + sigma2_sq + C2;
+   ssim_map = ones(size(mu1));
+   index = (denominator1.*denominator2 > 0);
+   ssim_map(index) = (numerator1(index).*numerator2(index))./(denominator1(index).*denominator2(index));
+   index = (denominator1 ~= 0) & (denominator2 == 0);
+   ssim_map(index) = numerator1(index)./denominator1(index);
+end
+
+mssim = mean2(ssim_map);
+
+return