Return (large) unchange mxArray from MEX
10 views (last 30 days)
Show older comments
Hi all ..
With a large array out to CPP / MEX, under certain conditions there will be no manipulation of the original data. Can someone please provide an example of how to properly return this original array in the most efficient manner?
Essentially plhs[0] = prhs[0] logically.
Many thanks, Mark
0 Comments
Accepted Answer
James Tursa
on 27 May 2013
Edited: James Tursa
on 7 Nov 2013
plhs[0] = prhs[0] is perfectly OK in mex programming. MATLAB always (EDIT: NOT TRUE ... SEE BELOW) makes shared data copies of the plhs[ ] mxArrays and actually returns the shared data copies. To see this is true you can do this:
1) Compile this code (call it plhs_eq_prhs.c):
#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
if( nrhs ) {
plhs[0] = prhs[0];
mexPrintf("The mxArray structure address of the input is %p\n",prhs[0]);
}
}
2) Run the following commands at the MATLAB prompt:
>> format debug
>> a = 1:3
a =
Structure address = 46cdda0
m = 1
n = 3
pr = 16c3a440
pi = 0
1 2 3
>> b = plhs_eq_prhs(a)
The mxArray structure address of the input is 046CDDA0
b =
Structure address = 495aea0
m = 1
n = 3
pr = 16c3a440
pi = 0
1 2 3
You can see that the mxArray structure address of the variable b is different from the mxArray structure address of the input variable a, but the data pointers pr are the same. So MATLAB has returned a shared data copy back to the workspace.
You can get almost the same result by using the following:
mxArray *mxCreateSharedDataCopy(mxArray *); // supply the prototype
:
plhs[0] = mxCreateSharedDataCopy(prhs[0]);
However, in this case there will be an extra temporary shared data copy of prhs[0] created that will be destroyed when the mex routine returns to the caller, so it is slightly less efficient that doing plhs[0] = prhs[0] directly.
The general behavior of the MATLAB API functions (the documented ones) is:
- All of the mxCreateEtc type of functions create temporary mxArrays that are put on the Variable Array List (my terminology).
- When the mex routine returns control to the caller, MATLAB makes shared data copies of the plhs[ ] variables and returns those, then everything on the Variable Array List is destroyed.
- Using mexMakeArrayPersistent removes a mxArray variable from the Variable Array List. Once a variable is removed from this list, I am unaware of any function (documented or undocumented) that will get it back on this list. So it is up to you, the programmer, to destroy it at some point in your code or you will have a permanent memory leak that can only be recovered by quitting and restarting MATLAB.
- There is a companion function mexMakeMemoryPersistent for memory allocations made with mxMalloc and friends, but in this case there is an undocumented function you can use to get it back on the garbage collection list if you want.
- There are undocumented API functions that create mxArray variables. Some of them create temporary mxArray variables that are on the Variable Array List (i.e., scheduled for garbage collection), but others create normal variables that are not on the Variable Array List. So you need to be careful when using them to avoid permanent memory leaks. mxCreateSharedDataCopy is an example of one of these undocumented API functions ... it happens to create a temporary mxArray that is on the Variable Array List (scheduled for garbage collection).
3 Comments
James Tursa
on 7 Nov 2013
Ah yes ... I forgot about that case. Thanks for the correction. It is possible to pass in a temporary variable to a mex routine. You can get it as a field reference or cell element reference as you mention, but you can also get it with any other expression that you put in the argument list (e.g., adding two variables together). Apparently it can get treated differently in the plhs return logic, possibly related to the fact that it is not on the mex routine's garbage collection list. And, of course, there is no official way to detect this in a mex routine. So the bottom line is to use mxCreateSharedDataCopy as you suggest.
More Answers (4)
Jan
on 27 May 2013
Edited: Jan
on 28 May 2013
"plhs[0] = prhs[0]" is forbidden in MEX-programming, because re-using the inputs as outputs collides with the copy-on-write mechanism. Unfortunately the best (means documented and stable) method is to duplicate the array:
plhs[0] = mxDuplicateArray(prhs[0]);
And if it is your opinion, that this is not efficient for large array: Welcome to the team of MEX users, who ask TMW to document the methods for the creation of shared data copy. Please send an enhancement request to TMW.
You can try it with the undocumented mxCreateSharedDataCopy(). But a clean inplace processing would be much better.
[EDITED] James' explanation is correct and valuable. If you want the MEX to create an unchanged shared data copy of the inputs, plhs[0]=prhs[0] is valid. I've denied that, because this is a very rare need for a MEX function and I assumed, the contents of this variable must be changed inside the MEX. Then this can be observed:
// file: TestMex.c
#include "mex.h" // Must be called with a DOUBLE as input
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
plhs[0] = prhs[0]; // Ok until here
*mxGetPr(plhs[0]) = -1.0; // Don't do this
}
Compile it and run:
a = 1:4;
b = a; % Shared data copy inside Matlab
c = TestMex(b); % Modified shared data copy inside MEX
c % -1 2 3 4 as wanted
b % -1 2 3 4 !!!
a % -1 2 3 4 !!!
This means, that you plhs[0]=prhs[0] is valid, but modifications of the contents of plhs[0] are not and will destroy the integrity of the input values. Therefore performing this copy inside a MEX will not be useful for any real purpose.
What happens for:
format debug
a = 1:4;
a
a = TestMex(a)
format long g % clean up only
Now a is a shared data copy of itself. The Structure Address an pr are the same, but the increased reference counter might cause a memory leak. Unfortunately my trials to prove this have not been successful.
@Mark: I can unaccept my answer on demand.
7 Comments
James Tursa
on 3 Jun 2013
Edited: James Tursa
on 3 Jun 2013
@Jan: I pretty much am in agreement with you. The fact that the following statement
plhs[0] = (mxArray *) prhs[0];
overrides the const attribute of prhs[0], and the fact that the shared-data-copy method of returning plhs[ ] variables is not documented, is a bit unsettling as a programmer. However, I put it in the same category of using any of the other undocumented functions/features. E.g., I routinely use mxCreateSharedDataCopy because it has proven to be reliable and it works across multiple platforms and versions. No guarantee it will work in the future, but that doesn't stop me from using it ... the advantage is just too great not to in many cases.
Mark
on 27 May 2013
Edited: Mark
on 27 May 2013
4 Comments
James Tursa
on 25 Jan 2018
Another option that was just posted to the FEX for getting shared data copies of contiguous sub-sections of an existing variable:
See Also
Categories
Find more on Matrix Indexing in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!