How To Deblur The Image

Deblurring: The Details Matter
December, 2013
Mark DeNies DeNies Video Software
Abstract
In recent years there have been major advancements in solving the ill-posed blind deconvolution problem, that is: deblurring images where both the true image and the blur are unknown. While in the general case the unknown blur varies across the image, this paper deals only with the shift-invariant blur, i.e., where the blur remains constant throughout the image. This paper examines the deblurring method used by Levin et al. [8]1 and the variant of Babacan et al. [5]2 and finds that there are subtle changes in implementation that individually make small improvements, but collectively result in a significant advancement over the prior art. The improvements can be grouped in three categories: those that improve the quality of the result, those that improve the flexibility of the algorithms, and those that diminish the length of time to solve the problem.
special case where the PSF is uniform throughout the image. The PSF is normally describe as an image that represents how the light that should have fallen on the center pixel was somehow diverted to other pixels in the neighborhood while the shutter was open. This can occur because of a combination of factors: an imperfect lens, an out of focus image, and/or motion of the camera or the subject. The known blurred image is the result of convolving the unknown true image with the unknown PSF combined with noise, as described in equation (1). Other terms used in this paper are MAP x which is a maximum a-posteriori algorithm used to solve for the true image, given a known blurred image and a fixed PSF. Likewise, MAPk is a maximum a-posteriori algorithm which solves for a PSF given a fixed true image and a known blurred image.
Overview Nomenclature
The usual description of the blind deconvolution problem is expressed as follows: y=kx+n (1) Much of the recent research in blind deconvolution began with the work of Fergus et al. [3]. Fergus made the observation that natural images have a unique distribution of first derivatives, which can be approximated as a mixture of Gaussian distributions, and used as a prior to restrict the solution. Levin et al. [4] provides an analysis of why this technique was so successful. A number of others have built on these observations, such as Levin and Babacan. Yet other researchers have pursued different paths, concentrating on refinements of MAP (Cho [12]),
For the remainder of this paper, Babacan et al. [5] will be simplified to Babacan.
2
where y is the observed blurry and noisy image, k is the kernel or Point Spread Function (PSF), x is the unknown true (or sharp) image, and n is additive noise. This equation is applicable in the
For the remainder of this paper Levin et al. [8] will be simplified to Levin.
1
All contents are Copyright 2013 DeNies Video Software. All rights reserved
estimating the blur from sharp edges (Joshi et al. [11]), and reduction of artifacts (Shan et al. [7]). Babacan, in particular, demonstrated that the Mixture of Gaussian distribution was not really required: a simpler logarithmic model works just as well in practice, even though it doesnt match as closely to the expected gradient distributions. Levin introduced a MAPk algorithm which utilizes the covariances of the true image to solve for the PSF. This paper employs a variation of the technique used by Levin and Babacan and concentrates on the mechanics of solving the blind deconvolution problem rather than exploring new theory. This strategy produces several improvements in the quality of the results, as well as significant speed improvements. The speed improvements partially compensate for choosing to solve the MAP x problem in image space rather than the more common filter (or derivative) space. Quality improvements result from solving in image space using five derivatives, solving the MAPk problem without using the edge pixels of the true and blurred images, and introducing a novel stopping criteria in the conjugate gradient algorithm used in the MAPx algorithm. Time/flexibility improvement techniques include: Using the prior solution as the starting condition for the conjugate gradient algorithm, and improved resizing of the PSF and proposed true image to the next level when a multilevel solver is used. The new stopping criteria for the conjugate gradient also reduces the time required. Inspired by Kundar [6], this work applies another time/flexibility to make the solution easier to use: automatically determining the support (size) of the PSF. Proper sizing of the PSF at each level also leads to a reduction of the time to solve the problem.
Background
Up until the mid-2000s blind deconvolution techniques were based on the work of Richardson and Lucy [1, 2] or other Bayesian methods such as Total Variation (TV). Common to most blind deconvolution algorithms is an Expectation-Maximization (EM) solver which alternates between solving for the true image given the blurred image and the PSF and solving for the PSF given the blurred image and the most recent true image. These steps are repeated a fixed number of times reducing some control parameter (such as noise) at each step, or until the convergence criteria are met. The early Bayesian solvers typically included a sparsity prior and/or a Gaussian prior. However, these solvers often failed to find a good image, and suffered from ringing and noise amplification. Levin analyzed these early EM algorithms and concluded that these often tended to converge in the trivial solution (the true image being the same as the blurred image). Fergus et al [3] introduced two important concepts: first, creating a prior which enforces the statistics of natural images, and second, solving the problem in levels, progressing from low resolution to high resolution images. Levin further refined this technique, and replaced the MAPx,k with a non-blind solver, MAPx and a MAPk solver. This MAPk solver is well conditioned, since the known true and blurred images are large, and the unknown PSF, k, is small. An important feature of Levins MAP x solver is the selective use of masking when calculating ATAx to minimize ringing in the solution.
Description of the Algorithm

In equation (1) k, x and n are all unknown. Thus, equation (1) is a highly ill-posed problem with an infinite number of solutions. In order to solve this equation, additional constraints must be imposed. In the Bayesian framework, priors that model the statistics of natural images are used. A characteristic of natural images is that the first derivatives of the image are sparse, with most values being close to zero and few large values at sharp edges. The distribution is super-Gaussian and can be modelled as a mixture of Gaussian (MOG) distributions [3, 5, 8] or as a hyperLaplacian [9]. Figure 1: distribution of gradients. The vertical axis is the fraction of pixels with the absolute difference from its neighbors on the horizontal axis. Blue: blurred image Green: deblurred image Red: true image Magenta: Mixture of Gaussians
small, and working up to the full-sized image, the PSF and deblurred image is found at each level by alternately solving for the true image given a fixed PSF and blurred image, and solving for the PSF given a fixed true image and the blurred image. Formally: x = arg max p(x | y, k) k = arg max p(k | x, y) (2) (3)
When solving (2), this paper solves for the mean image of p(x | y, k) and calculates the covariance matrix Cx around it. The solution to (3) is: k = arg min kTAk k 2bkTk (4)
Given a PSF of size m by n, M = m*n and Ak is a matrix of size M by M. It contains the covariance of all m by n windows in x, and bk is the correlation of x with y. C(i,i) = diag Cx
p
(5)
Ak(i,j) = x(p+i)x(p+j) + C(p+i,p+j)

p
(6)
bk(i) = M x(n+i)y(n)
p
(7)
Where i,j are indexes in the PSF (i.e., 1M), and p sums the correlation across all image pixels.
Methodology
This article follows the successful strategy used in Babacan and Levin, and provides a summary of the algorithm. See their work listed in the references for a full description of the algorithm. Starting with a low resolution version of the blurred image where the blur is likely to be very Much of the methodology used in this work was created to minimize image edge effects. These can occur in all stages of working on the deblurring problem: when creating a synthetic blur, when making calculations throughout the solution of the deconvolution problem, and lastly when measuring the fidelity of the result.
Most researchers use synthetically blurred images to test their algorithms, and this paper uses them as well. This allows the resulting image to be numerically compared with the original using an objective measurement such as PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity), SSD (Sum-Square-Distances) or error ratio [4]. The first concern is the method used to construct a realistic blurred image. The method used in this work is to center the PSF, pad the image on all sides by half the size of the PSF, edge taper the image, and only then apply the blur. The
centering of the PSF prevents the process of blurring from also causing a shift. Since the deblurring algorithm solves for a true image which is equally padded on all sides by half the size of the blur, it is implicitly assuming a centered PSF. Padding and edge tapering are common techniques used to minimize the edge effects. In Table 1 below, the effects of these steps can be seen. The deblurred image created from the un-centered PSF is shifted down and slightly to the left compared to the original image. The PSNR is improved due to the improved symmetry of the blurred image.
Table 1: Effect of centering PSF before creating a blurred image (using Levins code) Original Image Blurred image Deblurred Image Un-centered PSF
PSNR: 28.11 SSIM=0.885 Centered PSF
PSNR: 28.90 SSIM: 0.896
In measuring the fidelity of the image (PSNR and SSIM), this work employs two techniques: First, only the center of the deblurred image is compared to the reference image, since the edge contains the effects of the padded area around the blurred image. Secondly, to allow for small shifts in the result, the image is matched to sub-pixel resolution. An interesting relationship is seen in the following table, showing how the PSNR of the first deblurred image varies with the amount of sub-pixel resolution: Table 2: Effect of Pixel resolution on PSNR Pixel PSNR resolution 1 33.881 0.5 33.881 0.25 34.588 0.125 34.651 0.0625 34.8079 0.03125 34.8079 0.015625 34.8083 0.0078125 34.8104 Sub-pixel resolution matching can result in up to a difference in measured PSNR of almost 1.0 over a match to the nearest pixel for the same pair of images. In this paper, the results will be shown for a pixel matching resolution of 0.0078125 pixels, and the search area will be 4 pixels in each direction. Another concern is whether the true image is actually free of blur. In the case of the test set used here (from Levin), it can easily be shown that the original image contains a small blur. Using the deblurring algorithm of Levin on their first test image the following images result:
Figure 2: Original image
Figure 3: Deblurred Original image and the discovered PSF (zoom in to see the small PSF):
The image of Figure 3 is noticeably sharper than the original image of Figure 2. The PSNR between the two is 33.9. When the algorithm in this paper is used to deblur the first image in the test suite of Levin (the first blurred image in the Appendix), the following image is obtained:
Figure 4: Deblurred Original Image
true image, and a novel stopping criteria is introduced. Simultaneous with solving for the true image, it calculates the covariances of the true images x and y gradients for use in the MAPk solver. The MAPk solver is the same as the one used by Levin and Babacan. However, only the interior of the blurred and true images are used to set up the quadratic problem to be solved. This excludes the problematic edge pixels from participating in the solution. Because of this, a somewhat lower sparsity weight is used than in Babacan, which in turn is significantly lower than in Levin. This will be discuss further in the section on Quality.
PSNR to Fig 2: 30.35 PSNR to Fig 3: 31.47 This image is closer to Figure 3, which indicates that the deblurring algorithm is removing most of the blur introduced by the synthetic blur as well as some of the latent blur in the original image. For this reason, when results are reported in this paper, the True image will be the deblurred original image. Yet another consideration is to be mindful of the edge effects in the deblurring in all of the algorithms used in blind deconvolution. This subject will come up often in the next two sections. Between each level, the PSF is resized, trimmed and centered. In the default case, the true and blurred images are doubled in area (1.414 in each dimension). But, typically the new PSFs dimensions will not be an exact multiple of 1.414. Centering the PSF to a sub-pixel resolution corrects for any drift at the current level. To accomplish both objectives, an intermediate PSF is created which is a multiple of the destination PSF, the PSF is scaled, centered, and then downsampled. For example, if the original PSF is 7x9, and the desired PSF for the next level is 11x13, the 7x9 PSF would first be resized to 99x127 (the closest integers to 14.14 times 7x9). The 99x127 PSF is placed in the center of a second 110x130 PSF, and then centered to the nearest pixel. The 110x130 PSF is then down-sampled by a factor of 10 to create an 11x13 pixel PSF accurately centered to within 1/10 of a pixel. Similarly, the deblurred image is resized for the next level, taking care to separately resize the core image and the padded border. More specifically, the padded border is removed, and the resultant core is resized by 1.414 in each dimension. The padded border is separated into the 4 sides and 4 corners. The sides are resized by ratio of the PSF sizes in the dimension toward the edge and 1.414 in the other. The corners are resized by the ratio of the PSF sizes.
Details of the Implementation

This work uses a multilevel solver in order to quickly determine the support and shape of the PSF. Within each level the noise parameter is reduced on each iteration and the algorithm alternates between a Bayesian MAP x solver to solve for the true image and a modified MAPk solver from Levin to solve for the PSF. The MAPx solver used in this work, unlike that of other researchers, solves in the image space using five derivatives. As is typical, a conjugate gradient algorithm is used. This conjugate gradient algorithm is initialized with the previous
In the Matlab source code associated with this paper, there are a number of parameters that are set in the top level which control various aspects of the solver. These include: Blind noise level: the noise level to be used in the solution of the multilevel process of finding the PSF. Default is .01. The non-blind noise level to be used in the final deconvolution of the image. Default: .007. The amount of image reduction at each level. Default: .7071 (each image is half size). The number of iterations in each level. Default: 15 A threshold value used to trim tiny PSF values when resizing the PSF. Default: .005 (pixels less than this are set to zero). The noise step size. Default: .88 The conjugate gradient stopping criteria. Default: stop when .00075 of the pixels change less than .01 from the prior iteration. The matlab implementation may be found here.
Using a fixed value of the residual as a stopping criterion is not altogether effective, which is why many researchers use a fixed number of iterations and rely on the noise parameter to control the enhancement of noise. To be safe, these researchers often use a larger noise parameter. To address this problem, this work introduces a stopping criterion which appears to be effective in exiting the conjugate gradient algorithm once the image details have first appeared. Doing so has the added benefit of significantly reducing the iteration count and allowing smaller noise parameters to be used. The stopping criteria is controlled by two parameters: MaxChange and MaxPCT. Only the center pixels in the image are examined: those that are a distance of at least half the PSF size from the edge. When less than MaxPCT of these pixels change more than MaxChange from the prior iteration, the conjugate gradient loop will exit. The default values are MaxChange = .00075, and MaxPCT = .01 (1 percent). Levin shows that a MAPk solution produces superior results. This technique was also used in the work by Babacan. The difference between these two implementations is that in Levin, the padded true image and its covariances are used along with the smaller blurred image to create the coefficients in the quadratic equation to be solved, and a larger value (.01) is used for the sparsity parameter (k_prior_ivar). Babacan uses a true image and covariance matrix the same size as the blurred image and a smaller sparsity paramenter (.001). Using the padded true image introduces information from the problematic edges of the true image. This introduces noise into the PSF, which is partially removed by a larger sparsity parameter. This papers implementation goes a step further than Babacan: the MAPk matrix and vector are built from the interiors of the true image and the blurred image. This allows a smaller sparsity parmeter to be used (.0005) which is smaller than that used by Babacan (.001). To illustrate the efficacy of these changes, Levins original code was modified to use the smaller
Quality Improvements
Improvements to the quality of the solution discovered in this work include employing a novel stopping criteria in the conjugate gradient algorithm, using only the interior pixels of the true image to solve for the PSF, and solving for the true image in image space rather than the more common solution in filter (derivative) space. In solving for the MAPx (the true image), care must be taken to bring out the detail, and yet avoid magnifying the noise. A characteristic of the conjugate gradient algorithm is that it spends the initial iterations enhancing the detail, and then later as it continues to change the image, it also enhances the noise. So, there is a balancing act between the number of iterations in the conjugate gradient solver and the specified noise level.
prior_ivar_k, and the smaller interior of the true and blurred images. This results in a significant improvement in detail as shown in Table 3. Table 3: Effect of using different sized areas in calculating the MAPk matrix and vector Blurred Image Levin Babacan Modified Levin Proposed
PSF
PSNR
26.30
28.72
29.10
31.47
PSF
PSNR
23.74
24.64
26.66
27.56
These results show the effects: the original Levin code, by using the larger areas (and including the true images boundary), has more noise in the PSF, even though it uses the larger sparsity parameter. Babacans results show the smallest amount of noise in the PSF, but the PSF is actually too sparse. The modified Levin example does better although there is still noise in the PSF, it is small and the PSF is sharper. The proposed
algorithm has some noise, which is between that of Babacan and the modified Levin, but it has a sharp PSF. This improvement is the result of the other modifications, such as a more complex resizing of the PSF and estimated true image between levels, and always using the prior iterations result when solving for the current iterations true image. Also, unli ke Levin and Babacan, the true image is solved in image space, 8
and like them, the PSF is solved in derivative (or filter) space.
Time/flexibility improvements
Several techniques were used to significantly reduce the execution time of the software: 1. The prior iterations true image is used as the conjugate gradients starting image. 2. The PSF and true image are accurately resized between levels. 3. Further speed improvements are incorporated into the calculation of the MAPk arrays. The first is to use the deblurred image from the previous iteration as the starting image for the current iteration. When this is done, the conjugate gradient function starts with an estimate that is closer to the final solution, and therefore will converge faster. This assumes that the stopping criteria works well. For the initial iteration at each level, the deblurred image from the prior level is used. In the MAPx solver the conjugate gradient solver is called three times. Most of the iterations occur in the first call. Using the first image of Levins test suite as an example, if the conjugate gradient solver is initialized with a constant image (e.g., gray), it typically takes 150 iterations for the three conjugate gradient solutions. Initializing with the blurred image cuts the number of iterations to 85. Initializing with the prior deblurred image reduces it further to 60. This is a significant time savings, allowing the slower solution in image space to be competitive with solving in filter (derivative) space. In the filter space solver of Levin, the maximum number of iterations (15) is reached about 80% of the time, and since there are two derivatives solved separately on each iteration, this results in approximately 80 conjugate gradient iterations per pass.
For this to be effective for the first iteration at each level, the deblurred image from the prior level must be properly resized. The deblurred image consists of the center core surrounded by padding. The core scales by the same factor as the blurred image between each level. But the padding scales according to the ratios of the PSF sizes. This might seem like a small difference, but it can significantly increase the number of iterations if the resizing is not performed properly. So, the true image is resized in 9 pieces: the core, the four sides, and the four corners. The PSF is resized as described in the section Details of the Implementation. Another, rather mechanical time improvement is to further enhance the already efficient correlation function, getCorAbDiagCov() of Levin by unrolling the loops. This saves 30% of the time spent by getCorAbDiagCov(), and reduces the execution time overall by 7%.
Multilevel Solver
Fergus [3] introduced a multilevel solver as well as the use of the distribution of image gradients to solve the blind deconvolution problem. Many researchers have adopted this technique in their work. The idea is that solving the low resolution images will find the gross details of the PSF, and these will be refined at higher resolution images. At each level, a number of iterations (10 or more) will be made to further bring out detail as the algorithm alternates between solving for the image (typically in gradient space) and solving for the PSF, gradually reducing the noise estimate to the final estimated noise. Babacan utilizes this technique and conveniently produces an image showing the progression of refinement of the PSF. The two figures below show the progress in the final two levels.
Figure 5: Next to last level
in the starting iteration of the next level, it is destroyed by the MAPk solver, and the detailed PSF is then re-discovered. Why? The answer is the high noise estimates which control how much of the image is to be considered as noise (and thrown away), leaving only the coarse image details for the MAPk solver to discover. The loss of PSF detail in the first iteration suggests two possibilities: could the lower levels be eliminated, and only the full resolution image be solved? Or could the initial noise level be reduced when iterating at each level? The answer to the first possibility is yes. The initial levels are wasted. Usings Levin s code with the first image in her test suite, and using an identity starting PSF, Table 4 shows the results with and without the initial four levels:
Figure 6: Last level
As can be seen in Figures 5 and 6, although the final PSF from the first image is resized and used
Table 4: Levins algorithm with and without using a multilevel solver 5 levels Last level only Deblurred Image
PSF
PSNR: 26.31 SSIM: .806 Time: 389s
PSNR: 26.45 SSIM: .810 Time: 257s
All contents are Copyright 2013 DeNies Video Software. All rights reserved 10
Table 5: Levins algorithm with fewer noise reduction iterations Iterations: 11 9 7 Deblurred Image
PSF on each iteration of final level
PSNR: 26.31 SSIM: PSNR: 26.34 SSIM: PSNR: 25.95 SSIM: PSNR: 25.68 SSIM: .806 Time: 389s .808 Time: 306s .800 Time: 249s .794 Time: 184s As for the second possibility: what happens if each level starts with a lower noise level and the number of steps at each level is reduced? For the example shown in Figures 5 and 6, if the first five iterations are eliminated, the starting PSF of Figure 6 would be closest to the final PSF of Figure 5. This sounds appealing, since much of the PSF is known from the prior levels, and only the details remain to be discovered. An additional observation is that the low resolution images have inherently less noise than the full sized image much of the noise is canceled out by averaging pixel values in the process of reducing the image. However, as shown in Table 5, anything below 911 iterations gives increasingly poor results. It appears that the PSF becomes overly sharp with a reduced size image. So is there any reason to use the multilevel solver? There are, in fact, two reasons. The first is when solving in image space. This will be shown in the next section. The second is that using a multilevel solver allows the software to determine the support for the PSF in a time-efficient manner, allowing the final step to be performed with an estimate of the PSF size which is appropriate (over-estimating the support is costly in execution time).
Image space vs. Gradient Space

Levin provides a number of solvers with which to experiment. In general their tests show that solving the problem in gradient space works best. Not only are the results more accurate, but the MAPx gradient solver can be terminated after 15 iterations. By contrast the MAPx image space solver must iterate 50 times. It should also be noted that Levin set the number of iterations for the final image space solver to 70. Deblurring the first test image (image1/kernel1) and the twelfth image) image2/kernel4 with the gradient space deblur algorithm and image space algorithm produces the following results:
Table 6: Progression of PSF at the final level using various algorithms MAPx/ Deblurred Image MAPk Gradient Space/ Gradient Space (Levin) PSF for each iteration Deblurred Image PSF for each iteration
PSNR = 26.31 Image Space/ Image Space (Levin)
PSNR = 23.74
PSNR = 25.16 Image Space/ Gradient Space (this work)
PSNR = 21.99
PSNR = 31.47
PSNR = 27.56
First, it is apparent that solving in gradient space is superior to solving in image space, as Levin observed. However, the progression of PSF solution with noise is much slower in the image space solver. In fact, using the multilevel solver
is absolutely required when solving in image space! In this work, it was decided to use an image space MAPx solver inside of each levels iteration loop.
The reason for this was that other researchers (Levin and Babacan among them) preferred to use the image space solver for the final deblur, using five derivatives for more accuracy. Since this results in a more accurate true image at each iteration, it was felt that a more accurate true image should also result in more accuracy in the solved PSF. But, solving in image space is slower. In this work, the slowness was partially rectified by initializing the MAP x image space solver with the prior iterations solution, thus eliminating the first set of conjugate gradient iterations. Then by adding a stopping criteria, which measures the rate of change in the interior of the true image, the MAPx solver exits well before the maximum number of iterations is reached. As shown by the results in the Appendix, this combination results in a superior solution. It seems that when solving for the true image in the filter space, the solution converges faster, but that solving in the image space, a better image is obtained. This only works if it is coupled with the superior derivative space solver for the PSF. Secondly, there is little carry-over of information from one level to the next in the filter space solver, since its solutions during the initial iterations of each level results in data which the MAP k solver uses to find a very Gaussian-like PSF (as mentioned in the section Multilevel Solver).
Quality Improvements section, Babacan, and this implementation. The Appendix shows a steady improvement in quality in the order: Levin Original, Babacan, Modified Levin, and this papers implementation. A surprise result was that there is a significant time improvement for the Modified Levin test over the Levin Original test.
Results on Real Images

This section compares the techniques of Levin, Babacan, Shan, and this work. The test images were taken from Krishnan et al. [9] and Cho [12]. Default control parameters were used for all tests. Table 8 shows that, overall, Levins original technique produces the images with the fewest artifacts, although the images are still a bit blurry (e.g. the Philips on image 3 is barely readable). The modified Levin code, which did much better on Levins test suite in the Appendix, fared poorly here. Babacans results are intermediate between the two. The results for Shan et al. [7] take less time (the algorithm is released as an executable, not as matlab code) and does well on these examples. The proposed algorithm creates sharper images (except for the sailboat), but has noticeable artifacts. Better results could have been found by experimenting with the parameters, which each technique allows.
Results on Synthetic Images

For comparison this paper uses the test suite of Levin. The Appendix shows the results for all 32 images in the suite using the following algorithms: the original Levin code, Levin modified with the changes described in the
Table 7: Real images (zoom in to see details). Note: Levin and Babacan were modified to deblur color images. Blurred
Levin
Time: 12021s Levin Modified
Time: 5427s Time: 5009s
Time: 4864s
Time: 5677s Babacan
Time: 3973
Time: 15259s
Time: 8959s
Shan
Time: 2122s Shan PSF
Time: 8246s
Proposed
Time: 12897s
Time: 4509s
Table 8 shows the results of deblurring the sailboat and flowers images with this papers algorithm using default parameters, and an alternate set of parameters. Unfortunately, this is a tedious process, and relies on the subjective judgment of the person deciding what is best.
In this case, four to six different trials were made with various combinations of higher sparsity and higher noise settings before choosing the ones in the second and fourth panels of Table 8. Essentially sharpness was traded for fewer echo artifacts in these images.
Table 8: Sailboat and flowers with alternate parameters Default Sparsity .002 settings Blind noise .04 Non-blind Noise .02 TrimPSFPct .02
Default settings
Sparsity .005 Blind Noise .014 Non-blind Noise .012
Time: 4509
Time: 3920s
Time: 6240s
Time:1334s
same diagonal tail appears. In fact, this same tail appears when the eight blurred images are deblurred regardless of the deblurring algorithms that were used, including: Shan et al., Cho, and others. As a second example of this effect, a non-blurry image with a strong diagonal pattern in a portion of the image was used (in this case, a fence). The entire image was deblurred, then only the portion without the diagonal feature was deblurred. The results appear in Table 9.
What doesnt work as well

Although the technique presented here performs well, there are blurred images for which the algorithm fails. The first example is when the original Levin images are deblurred to find a True image. The first three images in the Appendix, when deblurred, find reasonable, small blurs. But the fourth image has a significant diagonal tail when deblurred. Not so coincidently, this tail is in the same direction as the stripes on the childs shirt. In many of the deblurred fourth images in the Appendix, this
Table 9: Effect of strong features on the discovered PSF. Original Deblurred
PSF
Zoom in to see that the PSF for the first image has a noticeable diagonal. The PSF for the image with the fence has a faint tail in the same direction as the fence. Presumably this will also occur for more complex correlation patterns. It is even possible that hidden correlation patterns exist in the other three images in the test suite that account for some of the noise which is observed in the PSFs. A method to eliminate this effect is left to further research.
Conclusion
This paper has demonstrated that multiple small changes in the implementation of a deblurring algorithm can combine to result in significant improvements in the quality of the results and/or the time required to achieve a solution. Overall, the enhancements in this paper result in an average quality improvement as measured by PSNR from 26.26 for Levin, 27.12 for Babacan, to 29.20 for this papers implementation.
Future work
A natural follow-on to this work is to investigate how to overcome the false tail which was discussed in the section What doesnt work well. An unrelated observation is that some PSFs result in a deblurred image which is still noticeably blurry, no matter whose algorithm is used. A Gaussian blur is one of these. Why this occurs has not been addressed in the research as far as this author could find. As was noted in the section Results on Real Images, the default parameters which control the
deblurring process are, at best, a compromise. Better results for an individual image are often found with different settings for the parameters. This could be automated, but an algorithmic test of the goodness of the image is needed. There has been some research into this problem (especially in estimation of noise), but this author found the methods for estimating noise gave poor results for the images used in this paper. Another potential area of investigation is to apply the techniques used here to the general shiftvariant deconvolution problem.
Appendix
This appendix compares several algorithms using Levins test suite. This suite consists of four images blurred with eight PSFs as described in the Methodology section. In this work, the original image is also deblurred to discover a True image. The tests labeled Levin Original and Babacan were run using the original codes settings, but Babacans was modified to use Levins equivalent optimized correlation code to calculate the kernel at each step, since Babacans method ran out of memory on 32 bit machines for the larger kernels. Modified Levin is the original Levin code modified to use only the interiors of the true and blurred images in getCorAbDiag, and the k_prior_ivar (kernel sparsity constant) changed from .01 to .001, and w0 in deconvSPS.m (controls the amount of detail/noise) changed from .1 to .01. The following PSFs were applied to the Original image after being centered. The True PSF will be a combination of this PSF and the small blur inherent in the Original image. PSFs:
Image 1 Original True
Blurred
Levin Original
Modified Levin
Babacan
Proposed
PSNR: 26.31 SSIM: .806 398s PSNR: 29.10 SSIM: .901 320s PSNR: 28.72 SSIM: 0.91 547s PSNR: 31.47 SSIM: .943 307s
PSNR: 27.05 SSIM:.835 286s PSNR: 29.89 SSIM: .910 262s PSNR: 29.90 SSIM: 0.93 379s PSNR: 31.93 SSIM: .946 324s
PSNR: 23.97 SSIM:.741 889s PSNR: 26.75 SSIM: .838 581s PSNR: 26.65 SSIM: 0.87 1259s PSNR: 26.68 SSIM:.881 685s
Blurred
Levin
Modified Levin
Babacan
Proposed
PSNR: 24.50 SSIM:.763 379s PSNR: 27.02 SSIM: .863 307s PSNR: 25.29 SSIM: 0.83 532s PSNR: 28.61 SSIM: .908 426s
PSNR: 26.09 SSIM: .829 537s PSNR: 30.06 SSIM:.917 383s PSNR: 27.51 SSIM: 0.91 762s PSNR: 28.91 SSIM: .920 514s
Blurred
Levin
Modified Levin
Babacan
Proposed
PSNR: 27.66 SSIM: .869 643s PSNR: 31.75 SSIM: .935 429s PSNR: 28.67 SSIM:0.90 886s PSNR: 24.85 SSIM: .805 498s
Blurred
Levin
Modified Levin
Babacan
Proposed
PSNR: 25.52 SSIM: .793 379s PSNR: 26.38 SSIM:.842 301s PSNR: 25.55 SSIM:.82 495s
PSNR: 26.54 SSIM: .863 692s
PSNR: 24.00 SSIM: .756 332s PSNR: 24.42 SSIM:.784 268s PSNR: 21.95 SSIM: 0.66 429s PSNR: 24.88 SSIM: .819 632s
PSNR: 24.62 SSIM: .799 530s PSNR: 24.30 SSIM: .798 380s PSNR: 24.21 SSIM: .82 725s
PSNR: 25.35 SSIM: .862 807s
Averages for all 32 images:
PSNR: 24.47 SSIM: .785 620s PSNR: 20.79 SSIM: .579 431s PSNR: 19.28 SSIM: 0.47 888s PSNR: 23.64 SSIM: .758 635s PSNR: 26.26 SSIM: .822 478s PSNR: 28.37 SSIM: .876 352s PSNR: 27.12 SSIM: .86 657s PSNR: 29.20 SSIM: .899 569s
References
1. William Hadley Richardson. BayesianBased Iterative Method of Image Restoration, (1972). http://www.opticsinfobase.org/abstract.cfm?i d=54565. 2. L. B. Lucy. "An iterative technique for the rectification of observed distributions", 1974. Astronomical Journal 79 (6): 745754. Bibcode:1974AJ.....79..745L. doi:10.1086/111605. 3. Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman. Removing Camera Shake from a Single Photograph, 2006. http://cs.nyu.edu/~fergus/papers/deblur_ferg us.pdf 4. Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman. Understanding and evaluating blind deconvolution algorithms. Computer Vision and Pattern Recognition, 2009. http://www.wisdom.weizmann.ac.il/~levina/ papers/deconvLevinEtalCVPR09.pdf 5. S. Derin Babacan, Rafael Molina, Minh N. Do, Aggelos K. Katsaggelos. Bayesian Blind Deconvolution With General Sparse Image Priors, 2012. http://www.dbabacan.info/BDGSP.php 6. Deepa Kundur. A Novel Blind Deconvolution Scheme for Image Restoration Using Recursive Filtering, 1998. http://ik.itba.edu.ar/~jacoby/Image%20Proce ssing/Teoria/Clase7/Papers/a%20novel%20bl
ind%20deconvolution%20scheme%20for%2 0ir%20using%20recursive%20fi.pdf 7. Qi Shan, Jiaya Jia, Aseem Agarwala. High quality Motion Deblurring from a Single Image, 2008. http://citeseerx.ist.psu.edu/viewdoc/downloa d?doi=10.1.1.218.6835&rep=rep1&type=pdf 8. Anat Levin, Yair Weiss, Fedo Durand, William T. Freeman. Efficient Marginal Likelihood Optimization in Blind Deconvolution, 2011. http://www.wisdom.weizmann.ac.il/~levina/ papers/deconvLevinEtalCVPR11.pdf 9. Dilip Krishnan, Rob Fergus. Fast Image Deconvolution using Hyper-Laplacian Priors, 2009. http://cs.nyu.edu/~dilip/research/papers/fid_n ips09.pdf 10. Anat Levin, Rob Fergus, Fredo Durand, William T. Freeman. Deconvolution using natural image priors MIT 2007. http://groups.csail.mit.edu/graphics/CodedAp erture/SparseDeconv-LevinEtAl07.pdf 11. Neel Joshi, Richard Szeliski, David J. Kriegman. PSF Estimation using Sharp Edge Prediction, 2008. http://vision.ucsd.edu/kriegmangrp/research/psf_estimation/psf_estimation.p df 12. Taeg Sang Cho. Removing motion blur from photographs. PhD thesis, MIT 2010. http://people.csail.mit.edu/taegsang/Docume nts/PhDThesis_TaegSangCho.pdf

How To Deblur The Image

Transféré par

Informations du document

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

How To Deblur The Image

Transféré par

Droits d'auteur :

Formats disponibles

Deblurring: The Details Matter

Description of the Algorithm

Ak(i,j) = x(p+i)x(p+j) + C(p+i,p+j)

PSNR: 28.11 SSIM=0.885 Centered PSF

PSNR: 28.90 SSIM: 0.896

Figure 2: Original image

Figure 4: Deblurred Original Image

Details of the Implementation

Figure 5: Next to last level

Figure 6: Last level

PSNR: 26.31 SSIM: .806 Time: 389s

PSNR: 26.45 SSIM: .810 Time: 257s

PSF on each iteration of final level

Image space vs. Gradient Space

PSNR = 26.31 Image Space/ Image Space (Levin)

PSNR = 25.16 Image Space/ Gradient Space (this work)

Results on Real Images

Results on Synthetic Images

Time: 12021s Levin Modified

Time: 5427s Time: 5009s

Time: 5677s Babacan

Time: 4212s Time: 4267s

Time: 5918s Time: 5951s

Time: 2122s Shan PSF

Time: 4844s Time: 1427s

Time: 6240s Time: 11378s

Sparsity .005 Blind Noise .014 Non-blind Noise .012

What doesnt work as well

Table 9: Effect of strong features on the discovered PSF. Original Deblurred

Image 1 Original True

Image 2 Original True

Image 3 Original True

Image 4 Original True

PSNR: 26.54 SSIM: .863 692s

PSNR: 25.35 SSIM: .862 807s

Averages for all 32 images:

Vous aimerez peut-être aussi