(matlab coder)How should the coder::array member function set convert opencv's mat type to its type in-place?

20 views (last 30 days)
When using matlab coder to generate C++ code, the generator automatically generates a C++ class for coder::array, which is defined in the `coder_array.h` file (located in fullfile(matlabroot,'extern','include','coder','coder _array','coder_array_rtw_cpp11.h')), which has this usage note.
// set(T const *data, SizeType sz1, SizeType sz2, ...)
// : Set data with dimensions.
// : (Data is not copied, data is not deleted)
In addition, I have read in an image from an external Opencv and would like to convert it directly in situ using this set() member
cv::Mat srcImg = cv::imread("C:\\Program Files\\MATLAB\\R2022b\\toolbox\\matlab\\imagesci\\peppers.png");
// test cv::Mat to coder::array type directly
coder::array<unsigned char, 3U> result;
result.set((unsigned char*)srcImg.data, srcImg.rows, srcImg.cols,srcImg.channels());
But unfortunately the above conversion did not work and the result debugging found empty, how do I use it correctly?
Run in R2022b

Accepted Answer

cui,xingxing
cui,xingxing on 4 Mar 2023
Edited: cui,xingxing on 4 Mar 2023
After repeated investigations and trials over this period of time, in relation to my current problem, I have found out what the problem is and am sharing the exact answer in full.
Run R2022b, win10, test in mingw64 and vc complier.
Conclusion:
  • the member function set in the generated coder::array class is not able to convert opencv's Mat type to coder::array<typename T, int N> type in situ, unless you rewrite the inherited class yourself.
  • The entry-point function in Matlab, whether declared as `coder.rowMajor;` or specified directly on the command line using the `-codegen` argument `-rowmajor`, does not change the fact that the matlab array is still in coloum-major form (bug).
What can be done to solve this at the moment is to assign values element by element or to use memcpy to copy the data block form to do it.
-------
With test cases:
function outImg = mytest(inImg)%#codegen
% in = coder.typeof(uint8(0),[1000,1000,3],[1,1,0]);% :1000x:1000x3
% codegen -config:lib -c mytest -args in -lang:c++ -report
arguments
inImg uint8
end
outImg = imresize(inImg,0.5);
end
use above comment in command line, generated C++ code,test set() method :
// omit some code
// my custom function, use to convert opencv's mat to matlab matrix
void convertCVToMatrix(cv::Mat &srcImg, int rows, int cols, int channels,
unsigned char dst[])
{
CV_Assert(srcImg.type() == CV_8UC1 || srcImg.type() == CV_8UC3);
size_t elems = rows * cols;
if (channels == 3) {
cv::Mat channels[3];
cv::split(srcImg.t(), channels);
memcpy(dst, channels[2].data,
elems * sizeof(unsigned char)); // copy channel[2] to the red channel
memcpy(dst + elems, channels[1].data,
elems * sizeof(unsigned char)); // green
memcpy(dst + 2 * elems, channels[0].data,
elems * sizeof(unsigned char)); // blue
} else {
srcImg = srcImg.t();
memcpy(dst, srcImg.data, elems * sizeof(unsigned char));
}
}
int main(int, char **)
{
// The initialize function is being called automatically from your entry-point
// function. So, a call to initialize is not included here. Invoke the
// entry-point functions.
// You can call entry-point functions multiple times.
{
coder::array<unsigned char, 3U> inImg;
coder::array<unsigned char, 3U> outImg;
cv::Mat srcImg = cv::imread(
"C:/Program Files/MATLAB/R2022b/toolbox/matlab/imagesci/peppers.png"); // 384*512*3 size
// Since set_size() was set beforehand, then after set srcImg retains the array size of the set method, i.e. the prior set_size method is invalid! Also, if only set is used for the conversion, there is a dimension mismatch problem, i.e. it leads directly to the later calculation in the wrong dimension!
// inImg.set_size(384, 512, 3);
// inImg.set((unsigned char *)dst, srcImg.channels(), srcImg.cols,srcImg.rows);
// The following 2 sentences are the effective and correct way
inImg.set_size(384, 512, 3);
convertCVToMatrix(srcImg, srcImg.rows, srcImg.cols, srcImg.channels(),inImg.data());
// Call the entry-point 'mytest'.
mytest(inImg, outImg);
// output matlab matrix `outImg` to file,so i can read this matrix into matlab to show for debug.
std::cout << " outImg.size:" << outImg.size(0) << "," << outImg.size(1)
<< "," << outImg.size(2) << std::endl;
std::ofstream fid("outImg.txt", std::ios::out);
for (size_t i = 0; i < outImg.size(0); i++) {
if (i != 0) {
fid << std::endl;
}
for (size_t j = 0; j < outImg.size(1); j++) {
fid << (int)outImg.at(i, j, 0) << " "; // only see R channel array.
}
}
fid.close();
}
// Terminate the application.
// You do not need to do this more than one time.
mytest_terminate();
return 0;
}
// End of code generation (main.cpp)
Read the text file output from the above C++ code in matlab:
aa = uint8(readmatrix("outImg.txt"));figure;imshow(aa)
debug found that the only way to display the image correctly was to assign it element by element or my custom conversion function convertCVToMatrix() above.
In addition, due to space limitations above, I have tried specifying `coder.rowMajor` in the matlab entry function or `-rowmajor` on the command line, but this does not work, i.e. the resulting C++ code is still in coloum-major
However, thanks very much to @Alexander Bottema for his positive answer, even if it didn't solve the problem directly.

More Answers (1)

Alexander Bottema
Alexander Bottema on 23 Feb 2023
Edited: Alexander Bottema on 23 Feb 2023
The issue here is that MATLAB (and MATLAB Coder) uses column-major format for matrices. This means the first dimension is the one that traverses memory locations consecutively, whereas C usually uses row-major memory layout; the last dimension traverses consecutive memory locations. You can toggle between the two memory layouts by reversing the size vector. You can either do this manually, or automatically. Manually:
codegen -args { coder.typeof(uint8(0), [inf inf inf] } mytest
function im = mytest(im)
% You have size(im,1) == channels, size(im,2) == columns, size(im,3) == rows
...
and:
coder::array<unsigned char, 3U> result;
result.set((unsigned char*)srcImg.data, srcImg.channels(), srcImg.cols, srcImg.rows);
mytest(arr);
If the top-level input is in row-major format you can also annotate your function to use row-major, and thus you don't need to reverse the size vector in your head:
function im = mytest(im)
coder.rowMajor;
% You have size(im,1) == rows, size(im,2) == cols, size(im,3) == channels
...
and:
coder::array<unsinged char, 3U> result;
result.set((unsigned char*)srcImg.data, srcImg.rows, srcImg.cols, srcImg.channels());
mytest(arr);
  18 Comments
cui,xingxing
cui,xingxing on 25 Feb 2023
Edited: cui,xingxing on 25 Feb 2023
Again, I'd recommend that you don't return coder::array and instead use the pattern as shown above; i.e. pass the variables as a function argument by reference. I know that the example main code generation does return coder::array, but it is of historical reasons what we did for C code generation and that idiom got transferred to the C++ version. Note that all other code generation logic doesn't do this. It's only the example function that returns coder::array.
This is not a major issue, whether it is placed on a per-copy value return value or as a function reference to an output parameter, it has no bearing on the above issue.
---------
The reason why you see changes in pixel values has nothing to do with coder::array, set() or anything else. It's just the fact that the pixel data gets deallocated (somewhere) and the coder::array uses a dangling pointer.
I don't think the lack of change in image pixel values is irrelevant to coder::array for the following reasons:
use original set() method
static void argInit_d1000xd1000xd3_uint8_T(coder::array<unsigned char, 3U> &result)
{
cv::Mat dst;
cv::Mat srcImg = cv::imread(
"C:\\Program "
"Files\\MATLAB\\R2022b\\toolbox\\matlab\\imagesci\\peppers.png");
// or add result.set_size(srcImg.rows, srcImg.cols, 3); // Debugging up to this point is the constant 205 pixel value, a dangling pointer caused by coder::array ???
result.set((unsigned char *)srcImg.data,
srcImg.channels(),srcImg.cols,srcImg.rows);
}
The "result" variable is indeed correctly assigned a pixel value within the scope of the function, but once out of scope to an external function, the pixel value is unchanged (this time it's a 215 constant or some other value, i.e. a dangling pointer reference as you say)
Again, the downside of this approach is that you can't directly get the "result" variable correct dimension to pass to the external function.( for example, mytest(result) )
---------
Instead of doing this:
static coder::array<unsigned char, 3U> argInit_d1000xd1000xd3_uint8_T()
{
coder::array<unsigned char, 3U> result,temp;
cv::Mat dst;
cv::Mat srcImg = cv::imread(
"C:\\Program Files\\MATLAB\\R2022b\\toolbox\\matlab\\imagesci\\peppers.png");
unsigned char matlab_data[589824]; // numbers of pixels, peppers.png,384*521*3=589824
convertCVToMatrix(srcImg, srcImg.rows, srcImg.cols, srcImg.channels(),
matlab_data);
result.set((unsigned char *)matlab_data,srcImg.rows, srcImg.cols,srcImg.channels());
temp = result;
return temp; // This return value has the correct dimension and matlab type memory layout!
}
Do this instead:
static void argInit_d1000xd1000xd3_uint8_T(coder::array<unsigned char, 3U> &result)
{
cv::Mat dst;
cv::Mat srcImg = cv::imread(
"C:\\Program Files\\MATLAB\\R2022b\\toolbox\\matlab\\imagesci\\peppers.png");
unsigned char matlab_data[589824]; // numbers of pixels, peppers.png,384*521*3=589824
result.set_size(srcImg.rows, srcImg.cols, 3);
convertCVToMatrix(srcImg, srcImg.rows, srcImg.cols, srcImg.channels(),
result.data());
}
Don't rely on coder::array move semantics, especially when you wrap an existing data pointer.
Both of these methods work, except for a little difference in the form of the return argument, as both use my own converted memory method function convertCVToMatrix(),but it use deep-copy array,not in-place by set()
cui,xingxing
cui,xingxing on 25 Feb 2023
Therefore, it would be better to give direct examples of actual proofs of the set() method, in-place methods, in code that actually works! However, after trying hard above, I can only get it to work by copying element by element or by copying the whole memory block separately.

Sign in to comment.

Products


Release

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!