Acquire Data in Background Thread

Hello,

I am trying to create a data acquisition script that streams picoscope data. I do not have a lot of experience with MATLAB UI development, but I have expereince in other such as Python and Java; in these applications, I would offload the background data acquisition to a background/worker/separate thread with callbacks the UI. In my MATLAB application, I need to offload some data processing to a separate thread and the save to a file. It would be fine if the data acquisition happened in the main or worker thread. I have seen some funtions such as batch (Parallel Processing Toolbox), start(Data Acquisition Toolbox) and startBackground(Data Acquisition Toolbox), but I do not have acess to these toolboxes. Does anyone know of a way to accomplish this task? Thank you.

[hsmistry]

Hi Randall,

Which PicoScope model are you using and do you have access to Instrument Control Toolbox?

[Randall Reynolds]

Picoscope 4824. Yes, I have access to the instrument control toolbox. Do you know a way to lock stata in the bckground?

[Walter Roberson]

When you say a background thread, do you mean that it has to be in a different process, or at least a different pthread (POSIX Threads library), so that computation can happen at literally the same time (provided there are at least two cores) ?

Or do you mean that it would be acceptable if your current thread got automatically preempted sometimes while the data transfer happened?

If you need simulatenous execution then you would need to use Parallel Computing Toolbox (or hack it yourself using something like Java Threads to invoke a second MATLAB instance.) If automatic preemption is good enough, then it can typically be handled through callbacks.

[Randall Reynolds]

I would like it to be a part of the same process, just a different thread so the computation can happen at the same time; I have more than one core. I could not get the parrallel processing toolbox to work with the processing i needed. I thought about using a java thread, as I am used to this, but I would like to use as much MATLAB as possible. I would like to be able to create an interface and have some callback to when I determine the data needs to processed, but I have not found a way in MATLAB.

I have used MEX to process my code which speeds up my processing enough to handle 10 MS/s from the picosocpe.

[Walter Roberson]

Before R2020a, there was no user-controlled threading available. R2020a added thread-based parfor.

Otherwise, there is just:

• parfor, parfeval, batch (Parallel Computing Toolbox) -- uses a different process, data needs to be transferred between processes
• spmd (Parallel Computing Toolbox) -- uses a different process, data needs to be transferred between processes, uses a different data transfer mechanism than the others (message based)
• java threads (that you would have to use to start a different process; you cannot fork off a different control thread for MATLAB)
• callbacks (which preempt the existing MATLAB control thread)

I would like to be able to create an interface and have some callback to when I determine the data needs to processed

You could create listeners and notify() of events ?

https://www.mathworks.com/help/matlab/events-sending-and-responding-to-messages.html?s_tid=CRUX_lftnav

[Randall Reynolds]

Okay. parfor did not work because I needed information from earlier in the script. I do not remember the exact error though. I received a similar error with batch as well.

I didn't know you could create and assign listeners in MATLAB. I believe should help speed up my code and organize as well. This is what I was looking for.

[Walter Roberson]

If the needed data was from before the parfor started then parfor would normally copy the data in when it sees reference to it, but you can also be more specific with https://www.mathworks.com/help/parallel-computing/parallel.pool.constant.html which has potential performance improvements.

Data can be sent between client and worker using parallel data queues, in either pollable or nonpollable varieties https://www.mathworks.com/help/parallel-computing/parallel.pool.dataqueue.send.html

If you are using Mac or Linux then sometimes it is worth using the File Exchange contribution to do shared memory. However I suspect that has not been updated to take into account changes to representation of complex values in R2018a (I think it was) -- that changed the header representation so the changes affected even non-complex variables.

[Randall Reynolds]

Thanks for all the advice Walter!!