Transpose... Not Transposing!

Hello Community,

I have an annoying glitch in some code, in that there is a line in my script that tells some data that has just been created in a for loop and stored to a struct to transpose. However, sometimes it will transpose and the subsequent output from the script works, and other times it doesn't transpose, and I get thrown a "matrix dimensions must agree" error! The code is:

% Calculate mean for all VLM CA ranges in the 'ZS.' struct
for i = 1:N
ZS.vlmcamean(i) = mean(ZS.ZSFLCA.(sprintf('ZSca%d',i)));
end

% Transpose the VLM CA mean ('vlmcamean') field
ZS.vlmcamean = ZS.vlmcamean';

Where you can clearly see the crucial " ZS.vlmcamean' " to transpose the data. The reason I do this is to make the data be the correct size for a function that follows in the script.

Can anyone suggest a workaround or advise on how to have this transpose element consistently do what it is meant to?

Many thanks.

10B.

[KSSV]

Are you sure the error is due to transpose. Can you tell what are the dimensions when it works and when it throws error?

[the cyclist]

Can you post an example that we can run, that distills this into the simplest possible case that replicates the error? (I find that this process itself will often expose the problem.)

[10B]

Hello KSSV and The Cyclist,

The dimensions when it works are (26,1), and where it throws the error it is (1,26). The output would then be joined with another output to generate a (26,2) variable which is needed for the function.

I will try to replicate the data to post something that could be run.

Thanks,

10B.

Answer 1

[Image Analyst]

Please show the entire error message - all the red text, not just part of it. I find it hard to believe the

ZS.vlmcamean = ZS.vlmcamean';

would throw an error. It's just simply transposing a row vector into a column vector. Let me see the entire error message. Also put these line before that line

rowVector = ZS.vlmcamean;
whos rowVector

The workaround is to just use reshape()

ZS.vlmcamean = reshape(ZS.vlmcamean, [], 1); % Make into column vector.

[10B]

Hello Image Analyst,

Here is the info you need:

Name            Size            Bytes  Class     Attributes

    rowVector      26x1               208  double              

Error using horzcat
Dimensions of matrices being concatenated are not consistent.
Error in ValidationScript (line 310)
vlm = [ZS.vlmcamean ZS.vlmcasd];

Now, I have changed the script for two 'transpose' sections, with your reshape suggestion, and it does appear to work consistently now which is great.

Do you have any idea/suggestions as to why the script worked on some occasions and not others? Just so I can add it to the knowledge bank and try to avoid the same in the future.

Cheers,

10B.

[Image Analyst]

See, now we know that the error is in this line:

vlm = [ZS.vlmcamean ZS.vlmcasd];

not this line:

ZS.vlmcamean = ZS.vlmcamean';

I bet ZS.vlmcamean was a column vector and ZS.vlmcasd was a row vector or vice versa.

Anyway, it sounds like (from your comment to dpb) that you have this all figured out and fixed now.

[10B]

Well, yes it did throw the error on the vlm = [ZS.vlmcamean ZS.vlmcasd]; line, but this was because ZS.vlmcamean = ZS.vlmcamean'; hadn't transposed so the vlm variable wasn't the right shape. The not transposing correctly, I believe, was the root of the problem.

Anyway, with yours, DPB and all the other folks inputs, it appears to be working fine now, so thanks again.

[dpb]

"...because ZS.vlmcamean = ZS.vlmcamean'; hadn't transposed"

NONSENSE! Set a breakpoint and you'll find it'll transpose every time as advertised. Now, as described above, you'll find that if you don't recreate the array from scratch every time, the next time you run the script it'll be the direction it was left in after the previous run and you'll transpose it again from "right" to "wrong". But you will NOT find a case where the result of transpose doesn't do what it says.

ADDENDUM

A sample script that emulates what you're doing that illustrates the problem is easily generated--

>> clear x  % make sure don't have a copy hanging around first time
for script=1:5  % simulate rerunning a script multiple times w/o *clear*
  for i=1:5, x(i)=randn(1);end     % create an array dynamically
  x=x.';                           % and transpose it
  if isrow(x)  % display which orientation it is after transpose...
    disp('X is row')
  else
    disp('X is column')
  end
end
X is column
X is row
X is column
X is row
X is column
>>

OK, the orientation is flipped every pass...after the initial transpose it's a column, hence as noted before the default when dynamically allocating is a row. Now the second and subsequent passes simply address the existing array in whatever orientation it is; Matlab doesn't care it uses linear addressing either way. But, every pass transposes so every other time it's a row vector instead of a column; precisely your symptoms that you created by not paying close attention to what you had done.

And, to illustrate the point of creation vis a vis access is the issue, here's the same pseudo-script except with the clear x statement moved inside the outer loop (script repeat simulation)--

>> for script=1:5,clear x;for i=1:5,x(i)=randn(1);end,x=x.';if isrow(x),disp('X is row'),else,disp('X is column'),end,end
X is column
X is column
X is column
X is column
X is column
>>

Now everything is as expected....

Answer 2

[dbmn]

I suggest to consider seeing the problem from another perspective.

Basically what you want is not the transpose of your input but the "right" format of it (and the best part is, that you already know how it should look like). So my suggestion is to use something like

ZS.vlmcamean = reshape(ZS.vlmcamean, [max(size(ZS.vlmcamean)), min(size(ZS.vlmcamean))]);

Maybe due to some unforeseen reason your ZS.vlmcamean looks differently in the loop. And the solution will make it always look the same.

[10B]

Thanks dbmn, a good suggestion to look from this perspective which is essentially what I need to acheive.

I have run your suggestion and it returns another error:

Error using horzcat
Dimensions of matrices being concatenated are not consistent.
Error in ValidationScript (line 305)
vlm = [ZS.vlmcamean ZS.vlmcasd];

which is at the point that the two variables are combined to make the (26,2). Again, the column seems not to have worked, but thanks for the ideas...

Answer 3

[dpb]

I'm quite certain Matlab is not "not transposing"; symptom sounds to me as though likely you're alternating operating on a row or column vector depending upon which was last left in the workplace the previous iteration; Matlab will retain existing array orientation when addressing it with single subscript.

Either be sure to clear the new accumulator when starting fresh so the default behavior will occur each time or alternatively, you could ensure the orientation by adding the second dimension subscript explicitly.

for i=1:N
  newVar(i,1)=whatever;   % create as column vector by dynamic allocation
  ...

newVar=zeros(N,1);        % preallocate first as column vector (recommended)
for i=1:N
  newVar(i)=...           % address existing column vector

Doing this will ensure orientation and as side benefit eliminate an extra transpose step plus is much more efficient to preallocate memory.

[10B]

Hello DPB,

I have run your version, and still got an error as below:

Error using rsquare (line 60)
Y and F must be the same size
Error in ValidationScript (line 325)
[r2 , rmse] = rsquare(flddata(:,1), vlm(:,1));

and upon checking the key variable(s), they are still (1,26) instead of (26,1)...

I still don't understand though how it wont work on the first call, but then does if I run it again?

I will keep the preallocation in the script though, a good reminder. Thanks.

[dpb]

As said, because by default ML in loop will create the row vector but if it exists after you've transposed it, will retain that shape which you then transpose again.

The above will definitely create a column vector unless and until you change it again somehow but you didn't show the code as modified nor the input data so can't tell where you went wrong...use the debugger and step through and see where you have the logic error.

[10B]

Ah yes, of course. Sorry for asking the same question again, I was trying to respond to everyone's helpful comments as they came in. Thanks for the help though, I think this one is sorted now.

10B.

[Nancy Hammond]

I'm having same problem.

WHEN I RUN LINES 374 AND 400 SEPARATELY:

line 374 sets rnt = log(1+cumret_a):

rnt = log(1+cumret_a);

size(rnt)

ans = 1 90

line 400 is function of transpose of rnt:

svt_x = [ NaN; -vt(2:end)+vt(1:end-1)+rnt(2:end)'-inflt(2:end)-grt(2:end)];

size(rnt)

ans =1 90

BUT GET ERROR WHEN RUN ENTIRE CODE:

Error in get_data_and_run_var_nh (line 402)

svt_x = [ NaN; -vt(2:end)+vt(1:end-1)+rnt(2:end)'-inflt(2:end)-grt(2:end)]; % use to check

identities.

>> size(rnt)

ans = 90 1

This occurs multiple times in this code from a completed paper by a prominent academic. Example:

line 88 omv_a = omv_a'; 90 by 1

line 166: ovy_a = omv_a./py_a; size 90 by 90 because

Now omv_a is 1 by 90, while py_a is 90 by 1

[Image Analyst]

@Nancy Hammond, can you please attach your data and code in a new question? Put the variables into a .mat file and attach with the paper clip icon.

save('answers.mat', 'cumret_a', 'rnt', 'vt'); % etc. for all the variables we need to run those lines of code.