inpaintExemplar

Restore specific image regions using exemplar-based image inpainting

Description

example

J = inpaintExemplar(I,mask) fills specific regions in the input image using the exemplar-based inpainting method. mask is a logical image that denotes the target regions in the image to be filled using inpainting.

example

J = inpaintExemplar(I,mask,Name,Value) specifies additional inpainting options using one or more name-value arguments.

Examples

collapse all

Read an image into the workspace.

I = imread('liftingbody.png');

Display the image.

figure
imshow(I,[])

Use the drawellipse function to select an elliptical region of interest (ROI) for inpainting. Use the 'Center' and 'SemiAxes' name-value pairs to specify the location of an ROI.

h = drawellipse('Center',[410 155],'SemiAxes',[95 20]);

Use the createMask function to generate a mask from the selected ROIs.

mask = createMask(h);

Display the image to be inpainted and its corresponding mask image.

montage({I,mask});
title(['Image to Be Inpainted','   |   ','Mask for Inpainting'])

Remove objects in the ROI by using inpainting.

J = inpaintExemplar(I,mask);

Display the original image and the inpainted image.

montage({I,J});
title(['Image to Be Inpainted','    |    ','Inpainted Image']);

Read an image into the workspace.

I = imread('forestdistorted.png');

Display the image. The image comprises distorted regions to be restored using inpainting.

figure
imshow(I,[])

Read a binary mask image containing the distorted image regions into the workspace.

mask = imread('imagemask.png');

Display the image to be inpainted and its corresponding mask image.

montage({I,mask});
title(['Image to Be Inpainted','   |   ','Mask for Inpainting'])

Inpaint the original image to restore the distorted image region. Specify the fill order and the patch size for inpainting as tensor and 7, respectively.

J = inpaintExemplar(I,mask,'FillOrder','tensor','PatchSize',7);

Display the original image and the inpainted image.

montage({I,J});
title(['Image to Be Inpainted','   |   ','Inpainted Image'])

Input Arguments

collapse all

Image to be inpainted, specified as a 2-D grayscale image or an RGB image of size m-by-n.

Data Types: single | double | int8 | int16 | int32 | uint8 | uint16 | uint32

Spatial mask of target regions, specified as a 2-D binary image of the same size as the input image I. The nonzero pixels in mask specify the target regions to be filled using inpainting.

Data Types: logical

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: J = inpaintExemplar(I,mask,'FillOrder','gradient')

Filling order, specified as the comma-separated pair consisting of 'FillOrder' and 'gradient' or 'tensor'. The filling order denotes the priority function to be used for calculating the patch priority. The patch priority value specifies the order of filling for the image patches in target regions.

Data Types: char | string

Size of the image patch, specified as the comma-separated pair consisting of 'PatchSize' and one of these options.

  • A scalar, s — The image patch is a square region of size s-by-s.

  • A vector of form [p s] — The image patch is a square or rectangular region of size p-by-s.

The default image patch size is 9-by-9. An image patch refers to the image region considered for patch matching and inpainting.

Note

  • The size of the image patch must be at least 3-by-3 and always less than the size of the input image.

Data Types: double

Output Arguments

collapse all

Inpainted image, returned as a 2-D grayscale image or an RGB image of the same size and data type as input image I.

Algorithms

The exemplar-based image inpainting algorithm is a patch-based approach that restores target regions in the input image by using these steps.

  1. Identify target regions from the input image.

  2. Generate a binary mask of the same size as the input image. The nonzero pixels in the mask image must correspond to the target regions to be inpainted.

  3. Identify the source region. All regions, excluding the target regions, in the input image comprise the source region. That is, source region = input imagetarget regions.

  4. For every patch of size p-by-s centered on a boundary pixel in the target region, compute the patch priority by using the gradient or tensor method.

  5. Find the patch with the maximum priority. This patch constitutes the target patch to be inpainted.

  6. Given the target patch, search for the best-matching patch in the source region by using the sum of square difference (SSD).

  7. Copy image data from the best-matching patch to the target patch.

  8. Update the input image, mask, and patch priority value.

  9. Repeat steps 4–8 until the target regions are inpainted.

References

[1] Criminisi, A., P. Perez, and K. Toyama. "Region Filling and Object Removal by Exemplar-Based Image Inpainting." IEEE Transactions on Image Processing. Vol. 13, No. 9, 2004, pp. 1200–1212.

[2] Le Meur, O., M. Ebdelli, and C. Guillemot. "Hierarchical Super-Resolution-Based-Inpainting." IEEE Transactions on Image Processing. Vol. 22, No. 10, 2013, pp. 3779–3790.

Introduced in R2019b