Continue displaying output using disp() function

I use disp() in a live script function to track which samples in a data set have been processed in a parfor loop. However, when long enough, the disp() out put will be something like this:

Sample 300 has been processed.

Sample 301 has been...

and will no longer output.

I've looked around and can't seem to how to remove an output limitation. Is there a way to increase/unlimit this or is it immutable?

[Sindar]

hmm, I can't seem to replicate this in 2020a.

for ind=1:1e3
    disp("Sample " + ind + " has been processed.")
end

prints all 1000 lines (you have to scroll a bit). So does

disp("Sample " + (1:1e3)' + " has been processed.")

Can you create a minimal example?

[Douglas Miller]

Sure!

for ind=1:10000

disp(['Sample ', int2str(ind), ' has been processed'])

end

recreates the issue

[Sindar]

So it does! Mine gets to

Sample 1971 has been processed
Sample...

While I think about it, what version do you have and is this the same point that yours stops?

[Sindar]

Notes:

for ind=1:10000
    fprintf('Sample %d has been processed\n',ind)
end

ends at exactly the same point.

for ind=1:10000
    fprintf('Sample %d\n',ind)
end

ends at

Sample 5092
Sam...

meaning the limit is related to characters, not lines

[Sindar]

So, this might not solve your problem (due to the parfor), but my method for tracking progress is this:

N = 20;
fprintf('%s\n',repmat('o',1,N))
for ind=1:N
    % do stuff
    fprintf('.')
end

which displays a fairly nice progress bar in the normal command window:

oooooooooooooooooooo
...............

though livescripts split the output, so it's a little harder to compare.

When I have too many iterations to show on one line, I cut it up and get something like:

Running 100 calculations
/5 oooooooooooooooooooo
1  ....................
2  ....................
3  ....................
4  ....................
5  ..............

This may be a solution if we can't figure out a way around the cut-off, and you don't need to know which samples have completed, just how many

[Douglas Miller]

Interesting observation! I find this in both 2019b and 2020a. I've changed my search term from "line limit" to "character limit" and this seems to be immutable according to https://www.mathworks.com/matlabcentral/answers/73092-what-is-the-fprintf-size-limit

Thanks for the help!

[Sindar]

I don't think this is the same problem since we are far below that limit (~6e4 characters) and that has to do with printing to files vs livescript output. Also, note that running the test in the command line prints everything for me.

It seems most likely that your issue is butting against the livescript's intent to avoid doing things like accidentally outputting huge arrays. But, it seems like there should be a way around it

Answer 1

[K.]

In this case, your code is hitting a 60,000 character limit per output. This particular limit is part of the Live Editor, and unfortunately, there is no way to change it. Our development team will take this as feedback, and we hope to address it in a future release.

That being said, there might be some ways you can work around this limit. Although I’m not sure they’ll meet your needs, as the first two don’t work with parfor.

Workaround 1:

The Live Editor automatically groups textual output from the same line into a single output. That means that if we introduce some other output on a different line, we can have multiple outputs, each with their own 60,000 character limit. Here is a crude example that demonstrates this. The output doesn’t look so nice, but you can see all the progress.

for ind=1:10000
    if (mod(ind, 1000) == 0)
        disp('--')
    end
    disp(['Sample ', int2str(ind), ' has been processed'])
end

Workaround 2:

Some people have used the ‘\b’ (backspace) character to create textual progress indicators (e.g. https://blogs.mathworks.com/loren/2007/08/01/monitoring-progress-of-a-calculation/). I believe there are some tools for this on File Exchange, but as an example, you could do something like the following. Once again, it is a crude example only meant to demonstrate the concept.

for ind=1:20
    pause(.1)
    printStatus(ind) % Do the fprints in a local function so that all the textual output is coming from one line in the script.
end
function printStatus (ind)
    if ind == 1
        % The very first time we don't need to delete the old text
        fprintf('Last processed sample: %5d', ind);
    else
        % Each \b removes one character from the previous fprintf.
        fprintf('\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\bLast processed sample: %5d', ind);
    end
end

Workaround 3:

Since the textual output for the entire parfor is treated as one output and hence subject to the 60,000 character limit, you could potentially split up your work into a few parfors on separate lines:

parfor ind=1:1000
    doStuff(ind);
end
parfor ind=1001:2000
    doStuff(ind);
end
function doStuff(ind)
    disp(['Sample ', int2str(ind), ' has been processed'])
end

Not a great solution, but sharing just in case.

I hope this helps, and as I said earlier, we hope to address this in a future release.

[Douglas Miller]

This was great! Thanks for the information!