How to Return a Opencv image cv::mat to Matlab with the Mex C++ API

67 views (last 30 days)
Iam currently using the Mex C++ API to take a picture with a Basler camera (Pylon API) and send it back to Matlab. I have some issues with converting an opencv cv::mat data type to a suitable type for Matlab. The solutions provided by Matlab (https://de.mathworks.com/help/vision/ug/opencv-interface.html) for converting opencv data types unfortunately only work with the older mex c api, so I cannot use them.
I have managed the following workaround. This workaround uses two loops two assign the values to a matlab::data::TypedArray. This solution is very slow due to the two loops and the copy of the values. Can someone recommend another way which is faster or works without loops? See the extract code from my c++ mex file with ne mex c++ API:
#include <opencv2/core/core.hpp>
#include <opencv2/opencv.hpp>
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <chrono>
#include <string>
class MexFunction : public matlab::mex::Function{
public:
MexFunction(){}
void operator()(ArgumentList outputs, ArgumentList inputs) {
CGrabResultPtr ptrGrabResult
... Here I take a picture with the camera and save it to "ptrGrabResult"...
Mat openCvImage;
CImageFormatConverter formatConverter;
CPylonImage pylonImage;
// First convert the "ptrGrabResult" to a pylonImage
formatConverter.Convert(pylonImage, ptrGrabResult);
// Convert it to a openc picture
Mat openCvImage = cv::Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(), CV_8UC1,(uint8_t *)pylonImage.GetBuffer(), Mat::AUTO_STEP);
const size_t rows = openCvImage.rows;
const size_t cols = openCvImage.cols;
matlab::data::TypedArray<uint8_t> Yp = factory.createArray<uint8_t>({ rows, cols });
for(int i = 0 ;i < openCvImage.rows; ++i){
for(int j = 0; j < openCvImage.cols; ++j){
// [Row][Coloumn] = (row, column)
Yp[i][j] = openCvImage.at<uint8_t>(i,j);;
}
}
outputs[0] = Yp;
}
}
The problem with this solution is that it takes way too long for me. Is there a faster and convenient method to cast the cv:mat type without a loop?? Or maybe it is possible to use pointer in this case, but I was not able to implement this...

Answers (2)

cui
cui on 19 Sep 2022
Edited: cui on 19 Sep 2022
you can use opencv ptr pointer to convert cv::Mat / matlab matrix each other, for example, Matlab matrix conert to opencv Mat type as follows:
// convert matlab matrix to opencv Mat,and vice versa, fishEyeImg is Matlab image matrix,oriImg is cv::Mat
cv::Mat oriImg;
bool is3Channels = fishEyeImg.getDimensions()[2] == 3;
if (is3Channels)
{
oriImg = cv::Mat::zeros(oriImgS, CV_8UC3);
for (size_t i = 0; i < fishEyeImg.getDimensions()[0]; i++) {
cv::Vec3b* data = oriImg.ptr<cv::Vec3b>(i);
for (size_t j = 0; j < fishEyeImg.getDimensions()[1]; j++) {
data[j] = cv::Vec3b((uchar)fishEyeImg[i][j][2], (uchar)fishEyeImg[i][j][1], (uchar)fishEyeImg[i][j][0]);
}
}
}
else
{
oriImg = cv::Mat::zeros(oriImgS, CV_8UC1);
for (size_t i = 0; i < fishEyeImg.getDimensions()[0]; i++) {
uchar* data = oriImg.ptr<uchar>(i);
for (size_t j = 0; j < fishEyeImg.getDimensions()[1]; j++) {
data[j] = (uchar)fishEyeImg[i][j];
}
}
}
References:

cui
cui on 20 Sep 2022
Edited: cui on 20 Sep 2022
perhaps these can help you if you use MATLAB2022a or later.
uchar *data= openCvImage.data;
matlab::data::TypedArray<uint8_t> Yp = factory.createArray({rows, cols},data,data+rows*cols,MemoryLayout::ROW_MAJOR);
Since R2022a, the "inputLayout" parameter syntax has been added to create arrays.
template <typename ItType, typename T>
TypedArray<T> createArray(ArrayDimensions dims,
ItType begin,
ItType end,
InputLayout inputLayout)
References:

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!