What is Guided Image Filtering?

Guided image filtering uses context from a separate image, known as a guidance image, to influence the output of image filtering. Like other filtering operations, guided image filtering is a neighborhood operation. However, guided image filtering takes into account the statistics of a region in the corresponding spatial neighborhood in the guidance image when calculating the value of the output pixel. The guidance image can be the image itself, a different version of the image, or a completely different image. You can perform a variety of image processing operations such as colorization, image matting, flash/no-flash denoising, and edge-preserving filtering, by choosing different guidance images.

A pixel in the output image is calculated using the neighborhood around the corresponding pixel in the input image and the same neighborhood in the guidance image.

Algorithm

When you filter image A using a guidance image, G, guided image filtering determines the value at pixel i in the output B using the value of pixel i and its spatial neighborhood in the input image A and the corresponding pixel and its corresponding spatial neighborhood in the guidance image G. If the guidance image G is the same as the input image A, the structures in the output image B remain the same as in the input image A. If the guidance image G is different from the input image A, structures in the guidance image G influence the output image B, in effect imprinting these structures on the input image A. This effect is called structure transference.

The output image B is, locally, a linear transformation of the guidance image G.

$b_{i} = x_{k} g_{i} + y_{k}, \forall i \in ω_{k}$

b_i and g_i are the ith pixels of B and G, respectively. x_k and y_k are the coefficients in a square window, ω_k, which the algorithm assumes to be constant.

The imguidedfilter function optimizes the weights of the linear transformation to minimize the error E between the input image A and output image B.

$E = \sum_{i \in ω_{k}} ({(x_{k} g_{i} + y_{k} - a_{i})}^{2} + ε x_{k}^{2})$

a_i is the ith pixel of A. ε is the regularization parameter that controls the degree of influence that the guidance image G has on the output image B , and is comparable to the degree of smoothing.

If you specify a high value for ε, the coefficient x_k must be small, and y_k must be similar to the value of the pixel a_i of the input image A, to minimize the cost function. These values of x_k and y_k reduce the degree of influence of the guidance image G and smooth the output image B more. On the other hand, if you specify a low value for ε, you can specify a larger value for the coefficient x_k, which increases the degree of influence of the guidance image G and smooth the output image B less. When using the imguidedfilter function, you can control the value of ε, by tuning the DegreeOfSmoothing name-value argument, and the degree of influence of the neighborhood on the filtering, by specifying the NeighborhoodSize name-value argument. The neighborhood size is the radius of the window ω_k.

Applications

You can perform a variety of image processing operations, such as colorization, image matting, flash/no-flash denoising, and edge-preserving filtering, by choosing different guidance images.

Colorization — If you have a grayscale image on which you have interactively scribbled desired colors for different parts of the image, you can color the entire image using the desired colors. Use the chrominance channels of the image as the input image A and the luminance channel as the guidance image G that influences structure.
Image Matting — You can enhance the alpha matte data of an RGB image so that it captures thin structures. Use the alpha matte as the input image A and the RGB image as the guidance image G that influences structure.
Flash/No-flash Denoising — Images captured without a flash are more prone to noise. You can denoise a no-flash image by using a corresponding image captured using a flash. Use the no-flash image as the input image A and the flash image as the guidance image G that influences structure.
For an example of flash/no-flash denoising using a guided filter, see Perform Flash/No-flash Denoising with Guided Filter.
Edge-Preserving Filtering — You can filter an image while preserving its edges by using the same image as the input image A and guidance image G.
For an example of edge-preserving filtering using a guided filter, see Segment Thermographic Image After Edge-Preserving Filtering.

References

[1] He, Kaiming, Jian Sun, and Xiaoou Tang. “Guided Image Filtering.” IEEE Transactions on Pattern Analysis and Machine Intelligence 35, no. 6 (June 2013): 1397–1409. https://doi.org/10.1109/TPAMI.2012.213.