ocrMetrics

Store OCR quality metrics

Since R2023a

Description

The ocrMetrics object stores optical character recognition (OCR) quality metrics, consisting of the character and word error rates, for a set of images. The evaluateOCR function returns the ocrMetrics object.

Creation

Create an OCR quality metrics object by using the evaluateOCR function.

Properties

DataSetMetrics — Data set metrics

table

Data set metrics, specified as a table with one row. The table contains OCR metrics aggregated over the data set. The table contains one column for each metric specified by the Metrics name-value that correspond to the metrics of the evaluateOCR function.

ImageMetrics — OCR metrics

table with N rows

OCR metrics, specified as a table with N rows, where N is the number of images in the data set. The table contains one column for each metric specified by the Metrics name-value of the evaluateOCR function. Each image metric returns a vector with one element for each image in the data set. The order of the rows matches the order of the images in the data set.

Examples

Evaluate Accuracy of OCR Model

This example shows how to evaluate the accuracy of an OCR model that can recognize seven-segment numerals on a dataset. The evaluation dataset contain images of energy meter displays that have seven-segment numerals in them.

Download and extract dataset.

datasetURL = "https://ssd.mathworks.com/supportfiles/vision/data/7SegmentImages.zip";
datasetZip = "7SegmentImages.zip";
if ~exist(datasetZip,"file")
    disp("Downloading evaluation dataset (" + datasetZip + " - 96 MB) ...");
    websave(datasetZip,datasetURL);
end

datasetFiles = unzip(datasetZip);

Load the evaluation ground truth.

ld = load("7SegmentGtruth.mat");
gTruth = ld.gTruth;

Create datastores that contain images, bounding boxes and text labels from the groundTruth object using the ocrTrainingData function with the label and attribute names used during labeling.

labelName = "Text";
attributeName = "Digits";
[imds,boxds,txtds] = ocrTrainingData(gTruth,labelName,attributeName);

Combine the datastores.

cds = combine(imds,boxds,txtds);

Run OCR on the evaluation dataset.

results = ocr(cds, Model="seven-segment");

Evaluate the OCR results against the ground truth.

metrics = evaluateOCR(results,cds);

Evaluating ocr results
----------------------
* Selected metrics: character error rate, word error rate.
* Processed 119 images.
* Finalizing... Done.
* Data set metrics:

    CharacterErrorRate    WordErrorRate
    __________________    _____________

         0.082195            0.19958   

Display accuracy of the OCR model.

modelAccuracy = 100*(1-metrics.DataSetMetrics.CharacterErrorRate);
disp("Accuracy of the OCR model= " + modelAccuracy + "%")

Accuracy of the OCR model= 91.7805%

Algorithms

• Error rates are a percentage of characters (words) in the input groundTruthTxt that have been incorrectly predicted in resultTxt.

• To compute the number of incorrect predictions to use in the error rate calculation, the function uses the Levenshtein distance, which is defined as the minimum number of edits (such as insertions, deletions, or substitutions) required to change one word (or sentence) into another one.

  Error rate = (S + D + I)/N where,

  • S — Number of substitutions

  • D — Number of deletions

  • I — Number of insertions

  • N — Maximum number of characters (words) between groundTruthTxt or resultTxt

Version History

Introduced in R2023a