Edge-preserving smoothing

Edge-preserving smoothing or edge-preserving filtering is an image processing technique that smooths away noise or textures while retaining sharp edges. Examples are the median, bilateral, guided, anisotropic diffusion, and Kuwahara filters.

Introduction

In many applications, e.g., medical or satellite imaging, the edges are key features and thus must be preserved sharp and undistorted in smoothing/denoising. Edge-preserving filters are designed to automatically limit the smoothing at “edges” in images measured, e.g., by high gradient magnitudes.

For example, the motivation for anisotropic diffusion (also called nonuniform or variable conductance diffusion) is that a Gaussian smoothed image is a single time slice of the solution to the heat equation, that has the original image as its initial conditions. Anisotropic diffusion includes a variable conductance term that is determined using the differential structure of the image, such that the heat does not propagate over the edges of the image.

The edge-preserving filters can conveniently be formulated in a general context of graph-based signal processing, where the graph adjacency matrix is first determined using the differential structure of the image, then the graph Laplacian is formulated (analogous to the anisotropic diffusion operator), and finally the approximate low-pass filter is constructed to amplify the eigenvectors of the graph Laplacian corresponding to its smallest eigenvalues.

Since the edges only implicitly appear in constructing the edge-preserving filters, a typical filter uses some parameters, that can be tuned, to balance between aggressive averaging and edge preservation. A common default choice for the parameters of the filter is aimed for natural images and results in strong denoising at the cost of some smoothing of the edges.

Iterative filters

Requirements of the strict edge preservation commonly limit the smoothing power of the filter, such that a single application of the filter still results in unacceptably large noise away from the edges. A repetitive application of the filter may be useful to reduce the noise, leading to the idea of combining the filter with an iterative method, e.g., the Chebyshev iteration and the conjugate gradient method are proposed in ^[1] for graph-based image denoising.

Due to the interpretation of the edge-preserving filters as low-pass graph-based filters, iterative eigenvalue solvers, such as LOBPCG, can be used for denoising; see,^[2] e.g., to accelerate the repeated application of the total variation denoising.

Edge-enhancing smoothing

Anisotropic diffusion generates small conductance at the location of the edge of the image to prevent the heat flow over the edge, thus making the anisotropic diffusion filter edge-preserving. In the graph-based interpretation, the small conductance corresponds to a small weight of an edge of the graph describing a probability of a random walk over the edge in the Markov chain on the graph. If the graph weight was negative, that would correspond to a negative conductivity in the heat equation, stimulating the heat concentration at the graph vertices connected by the graph edge, rather than the normal heat dissipation. While not-physical for the heat equation, this effect results in sharpening corners of one-dimensional signals, when used in graph-based smoothing filters, as shown in reference ^[3] that also provides an alternative physical interpretation using the wave equation describing mechanical vibrations of a mass-spring system with some repulsive springs.

Edge-preserving upsampling

Signal upsampling via the traditional interpolation followed by smoothing for denoising evidently distorts the edges in the original ideal or downsampled signal. The edge-preserving interpolation followed by the edge-preserving filters is proposed in ^[4] e.g., to upsample a no-flash RGB photo guided using a high resolution flash RGB photo, and a depth image guided using a high resolution RGB photo.

References

^ Tian, D.; Mansour, H.; Knyazev, A.; Vetro, A. (2014). Chebyshev and Conjugate Gradient Filters for Graph Image Denoising. IEEE International Conference on Multimedia and Expo Workshops (ICMEW). pp. 1–6. arXiv:1509.01624. doi:10.1109/ICMEW.2014.6890711.
^ Knyazev, A.; Malyshev, A. (2015). Accelerated graph-based spectral polynomial filters. 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA. pp. 1–6. arXiv:1509.02468. doi:10.1109/MLSP.2015.7324315.
^ Knyazev, A. (2015). Edge-enhancing Filters with Negative Weights. IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 14-16 Dec.2015. pp. 260–264. arXiv:1509.02491. doi:10.1109/GlobalSIP.2015.7418197.
^ Knyazev, A.; Malyshev, A. (2017). Signal reconstruction via operator guiding. SampTA 2017: Sampling Theory and Applications, 12th International Conference, July 3–7, 2017, Tallinn, Estonia. pp. 630–634. arXiv:1705.03493. doi:10.1109/SAMPTA.2017.8024424.

[1] Tian, D.; Mansour, H.; Knyazev, A.; Vetro, A. (2014). Chebyshev and Conjugate Gradient Filters for Graph Image Denoising. IEEE International Conference on Multimedia and Expo Workshops (ICMEW). pp. 1–6. arXiv:1509.01624. doi:10.1109/ICMEW.2014.6890711.

[2] Knyazev, A.; Malyshev, A. (2015). Accelerated graph-based spectral polynomial filters. 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP), Boston, MA. pp. 1–6. arXiv:1509.02468. doi:10.1109/MLSP.2015.7324315.

[3] Knyazev, A. (2015). Edge-enhancing Filters with Negative Weights. IEEE Global Conference on Signal and Information Processing (GlobalSIP), Orlando, FL, 14-16 Dec.2015. pp. 260–264. arXiv:1509.02491. doi:10.1109/GlobalSIP.2015.7418197.

[4] Knyazev, A.; Malyshev, A. (2017). Signal reconstruction via operator guiding. SampTA 2017: Sampling Theory and Applications, 12th International Conference, July 3–7, 2017, Tallinn, Estonia. pp. 630–634. arXiv:1705.03493. doi:10.1109/SAMPTA.2017.8024424.

[1]

[2]

[3]

[4]