MATLAB Answers

0

How to I pass off a Simulink application command syntax to MATLAB Job Scheduler on a local cluster?

Asked by Ewen Chan on 5 Sep 2019
Latest activity Commented on by Ewen Chan on 10 Sep 2019
I have a MATLAB/Simulink application that I normally would call from the MATLAB command window with:
app inputfile.m
And then it would run.
If I want to pass that off to the MATLAB Job Scheduler, how would I go about doing that?
I tried following the help documentation about using the cluster profile, creating the parallel job, and then creating the task, except that in the syntax for createTask(j, F, N...)
I tried to put in the "app inputfile.m" as the function into the createTask function and that didn't work.
What would be the better way for me to pass off "app inputfile.m" to MJS?
Your help is greatly appreciated.
Thank you.

  0 Comments

Sign in to comment.

4 Answers

Answer by Edric Ellis
on 6 Sep 2019

Where do the results from running app inputfile.m go? Does this create variables directly in the base workspace?
To use app together with createTask, you'd probably need to call it like this:
createTask(job, @app, 0, {'inputfile.m'})
However, that might not do what you want. It might work to use the batch command, a bit like this:
job = batch('app inputfile.m');
wait(job), load(job)
If that does work, then you can simply call batch multiple times.
An alternative might be to use parfor to run app in a parallel loop. But that depends a bit on how the outputs of app are produced...

  10 Comments

job = batch('app inputfile.m');
wait(job), load(job)
If I submit multiple jobs to MJS/the cluster/job scheduler; is there a way for me to have it automatically pull down the results when the job is finished while I do other things in the MATLAB session?
If I issue the wait(job), load(job) commands right away, it then goes and waits for the job to finish and then gets the results from the job. But I want it to be able to do that whilst I work on other things in the meantime/while the job is running.
What would be the best way to do that?
Thank you.
If you're interactively waiting for things to complete, then perhaps the Job Monitor might be the way forward - there you'll be able to see when things are complete.
Programattically - you can either call wait with a timeout, or check job.State to see if it is 'finished'.
This would mean that I would have to create a M-script that will submit the job, correct?
The other question that I also have was that in the example (re: submitting a job as batch, by right-click on the M-script file), it automatically assigns the ID.
Is there a way to do that programmatically? i.e. instead of:
job_n = batch('app inputfile.m', ...)
that it will automatically assign an ID to it without having to manually specify the ID.
How would I go about doing that?

Sign in to comment.


Answer by Jason Ross
on 5 Sep 2019

Are the parsim or batchsim commands applicable to your workflow? They are designed to parallelize simulations.

  12 Comments

It may be complaining because of duplicate paths or paths that don't exist? You don't mention your platform, but on Windows dealing with drive letters can be finicky, for example.
Also, is is issuing a warning or an error? In the case of a warning you might just want to ignore it for proof of concept type work, and return to it when you have things generally working.
Jason:
It was an error.
But rather than me defining an additional variable for the AdditionalPaths, I just did this:
job = batch('app inputfile.m', 'Profile', 'local', 'Pool', 0, 'AdditionalPaths', path);
and that seems to be working.
It'll take a little while for my app to run, so I'll just let it go, but it does look like that it is finally running.
I'll have to see if it will produce the outputs and also for me to be able to get them from the job, but at least it is starting the run, so this looks like it's promising progress.
Thank you.
Yes, I am running this on WIndows, and to make matters worse, I think that some of the paths might be a mixture of OSes.
But again, it looks like that with the batch submission syntax that I have now, it appears to be working, so it looks like that it was able to take the mixed OS syntax.
Thanks.

Sign in to comment.


Answer by Ewen Chan on 9 Sep 2019

Tangential question - my MATLAB R2015a installation is missing the mdce.bat from C:\Program Files\MATLAB\R2015a\toolbox\distcomp\bin.
That file is apparently required to start the mdce service so that I can start MJS.
How can I "backfill" that missing file?
I'm not really sure why it's missing, but it looks like that is needed to start MJS.
Once again, your help would be greatly appreciated.
Thank you.

  2 Comments

When you run the installer, make sure that MDCE is selected and it should be there. When you set up the MDCE installation it's a best practice to do an all products install so you can service any inbound request from any arbitrary user.
If you try and backfill files manually it can become a tedious nightmare very quickly -- the installer knows how to do "the right thing".
Ahhh....I might not have much in the way of control over that then because that's handled by the IT administration and the installers are only downloadable by IT, and not by/for end users.
Hmmm....bummer.
Let me talk with my IT to figure out what's going on with that.
Thanks.

Sign in to comment.


Answer by Ewen Chan on 10 Sep 2019

So the application ran and I was able to get the results that were written out by the application, but at the end, when MATLAB tries to load the results from the job, this is the error message that I get:
Error using parallel.Job/load (line 33)
Error encountered while running the batch job. The error was:
The task result was too large to be stored.
Caused by:
Error using parallel.internal.cluster.FileSerializer>iSaveMat (line 281)
Data too large to be saved.
Any ideas on how I might be able to resolve this and/or get around it?
Thank you.

  4 Comments

Show 1 older comment
Edric:
Being that my application has been designated by the company as a legacy application, and therefore; no further development work is being put into it, is that something that I can do as part of the job submission process or something that happens outside of the app?
This error state does not exist nor manifest when I run the app manually, so I suspect that it has something to do with how the 'local' cluster is handling the saves, which also suggests that there should be a way that I can change how the job submission is being handled in regard to the local cluster so that this won't be an issue any more.
My theory is that because I don't encounter this issue when I am running the app manually, but encounter this error when I use batch, therefore; it's gotta be something with how batch is handling it when submitting to the 'local' cluster profile.
Your help is greatly appreciated.
Thank you.
It is indeed to do with how the 'local' cluster is saving data. But that is derived from your user-level MAT file preferences. If you change your default MAT file version to v7.3, then (unless you specify otherwise), all MAT files created by MATLAB - including those used by the 'local' cluster - will be able to store >= 2Gb. The link I provided should give you instructions as to how to do that.
The reason you don't see this when running your application locally is that the results are returned to you in RAM. Any cluster type has to store the results of a job in some persistent storage somewhere so that you can pick them up later. 'local' uses MAT files on disk for this.
Edric:
Thank you.
Sorry, I had missed that.
I thought that I would have to edit my Simulink application code to enable that.
My apologies.

Sign in to comment.