Reduce time execution in a real time serial data reading

Hello everybody,

I'm carrying out an app that reads and plot data from sensors by serial Port. Data sent contains the values of 8 sensors plus power value and sample number. Size of the data can change because data of sensors values can changue as well. Data are sent every 0,5 s approximatly. I have to keep the data in a matrix (no problem), and plot in an axes.

If I just read values, time of each loop is fastier than the time that takes a new data to enter in the buffer, so I just have to wait for the new value, and then save it in a matrix. But If I try to plot the 8 values of the sensors, then the code takes longer than those half seconds, what makes that some data get missed. Which solution could I implement? I was thinking to save all the data but just plot one sensor in each iteration, so I'm plotting just one of each 8 values for each sensor. That's not important because after thousend of samples, visually it is impossible to see the difference, since the graph has the limit betwenn 0 and the n, where n can be +15000, but the importan thing which is save the data for processing after is performed.

But even with this last method, time of function is sometimes about 0,5 seconds and I get often an error in the matrix assignament because data collected from serial are no consistent. I was thinking play with the buffer because I could keep there more than one read, since the size is bigger than a packet of data.

The code without changes is the next one. Some ideas about how to get it? Thanks in advance.

function A = leerpuertoSerial(com,app)
        
        hold(app.UIAxes,'on');
        a = serial(com,'BaudRate',115200);
        fopen(a);
        A = zeros(10000,10);
        n = 1;
        i = 0;
        cleanup = onCleanup(@()myCleanupFun(a));
        ylim([0 800]);
        
        while n<2000
            
            if a.BytesAvailable > 0
                i=i+1;
                s =fscanf(a); 
                % I read the data packet which is something like this 1,20,343,234,234,234,543,543,543,343
                C = textscan(s, '%d%d%d%d%d%d%d%d%d%d','delimiter',','); 
                A(i,:) = cell2mat(C);
                %I plot just from the 3 value up to 10 th value (1 to 8 th sensor)
                plot(app.UIAxes,i,A(i,3),'x-b');
                plot(app.UIAxes,i,A(i,4),'x-b');
                plot(app.UIAxes,i,A(i,5),'x-g');
                plot(app.UIAxes,i,A(i,6),'x-k');
                plot(app.UIAxes,i,A(i,7),'x-m');
                plot(app.UIAxes,i,A(i,8),'x-c');
                plot(app.UIAxes,i,A(i,9),'x-b');
                plot(app.UIAxes,i,A(i,10),'x-r');
                %Legends makes really slow the code.
                %legend(app.UIAxes,'s1','s2','s3','s5','s6','s7','s8');
                
                drawnow;
            end
            n = n+1;
            flushinput(a);
        end
        hold(app.UIAxes,'off');
        msgbox('Proceso Terminado');
        xlswrite('OUTPUT',A);
end

function myCleanupFun(a)

  fclose(a);

end

Answer 1

[J. Alex Lee]

If instead of a loop, you use a timer (https://www.mathworks.com/help/matlab/matlab_prog/use-a-matlab-timer-object.html), you might have enough control to do what you want.

I think in general the plot is slow...if you don't need each plot to have a distinct symbol, maybe you can combine the plot into a single command. You might also save time by plotting once, and updating the XData and YData.

Another problem is that A seems to grow on every iteration, and this can slow down code especially if A is really large.

Isn't it redundant to use fscanf together with textscan? I don't know if it will affect execution time significantly, but how about fileread() instead of fscanf?

I think a better overall approach would be to decouple your data collector and displayer.

[Carlos Molina]

Thanks a lot for the answer.

I was trying to use XData and YData but it seems there is no such a propierty for UIAxes. Finally, I though to use a figure rather than the UIAxes and ploting there, and the process was quite fast, but unfortunatly after 4000 samples, some error came out.

Right now I'm working in another way. Instead of a loop and check every datapacket, I changed the protocol of the device, and I sent 5 packet and a terminator, and I use serial.BytesAvailableFcn with a callback function.

So what I do is to execute a function when the terminator is received, then I scan the buffer where there are 5 data packet, I save them and I print just one. In while, buffer is collecting data and I have time for perform the plots (5 times more time until the databuffer read than before) so I'm using the buffer to keep data until I'm able to read again. Size buffer is enough for this.

But I found a problem associated... With the previous system, I was using cleanup to be sure that if something happend and the function crashes (for example, serial port disconected or error in reading), I close the serial port and I save the data collected to these moment. But right know, I have just two function and any of them are executing all the time, so what I have is:

function a

%set the figure, open port, define BytesAvailableFcn

function b

callbackfunction to be executed when BytesAvailableFcn condition

So function a executes once, and function b executes when bytesavailablefcn, so If I set here the cleanup, it will be executed after the function a or b is completed (depending where I put it), but is does not mean than the program itself is completed

How could I relationate the cleanup function just with the crash of the program? o with a general function that contained the execution of both function.. I do not know If I'm being clear.

function  leerbuffer()

figure;
fig = gcf;
hold on;
grid on;
ylim([0 800]);
plot(0,0);

global A;
A = zeros(100,10);
global i;
i=1;
s = serial('COM3','BaudRate',115200); 
%cleanup = onCleanup(@()myCleanupFun(s));
if (s.Status == 'closed')       
    s.BytesAvailableFcn = {@leer,fig} ; %// note how the parameters are passed to the callback 
    fopen(s);
end

end

function leer(obj,event,fig)

%cleanup = onCleanup(@()myCleanupFun(obj));

%Read data is the figure is open
if ishghandle(fig)==1
  
      global A;
      global i;
      global color;
      cadena = fscanf(obj);
      flushinput(obj);
      C = textscan(cadena, '%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d','delimiter',',');
      C = cell2mat(C);
      xlim([0 i+5]);
      n = i;
      A(n,:) = C(1:10);
      plot(i,C(3),'x-b');
      plot(i,C(4),'x-g');
      plot(i,C(5),'x-m');
      plot(i,C(6),'x-r');
      plot(i,C(7),'x-y');
      plot(i,C(8),'x-k');
      plot(i,C(9),'x-b');
      plot(i,C(10),'x-b');
      n=n+1;
      A(n,:) = C(11:20);
      n=n+1;
      A(n,:) = C(21:30);
      n=n+1;
      A(n,:) = C(31:40);
      n = n+1;
      A(n,:) = C(41:50);
      n = n+1;
      i = n;
      drawnow;
      disp(toc);
      
  
else
    
    fclose(obj);
end
end

%function myCleanupFun(obj)
%  fclose(obj);

%end

[J. Alex Lee]

First things first, XData and YData are properties of the plotted objects, and Axes (and UIAxes) are ultimately children of a Figure (or UIFigure). So your comments on why it wouldn't work didn't make sense to me. Also, I now realize that you are keeping the plots you drew at each iteration rather than clearing them and keeping the same number of plotted objects. This is actually worse than just having each plot have lots of data points.

Stickign with the original method, does the following help at all? I think it would still slow down quite a bit as i reaches into the thousands

function A = leerpuertoSerial(com,app)

%         hold(app.UIAxes,'on');
app.UIAxes.NextPlot = 'add';
app.UIAxes.YLim = [0 800];
a = serial(com,'BaudRate',115200);
fopen(a);
A = nan(10000,10);
n = 1;
i = 0;
cleanup = onCleanup(@()myCleanupFun(a));
%         ylim([0 800]);
for sID = 8:-1:1
	PlotHandles(sID) = plot(app.UIAxes,(1:size(A,1)),A(:,i+2),'x-');
end
legend(app.UIAxes,PlotHandles,{'s1','s2','s3','s5','s6','s7','s8'});


while n<2000
	
	if a.BytesAvailable > 0
		i=i+1;
		s = filread(a);
		% I read the data packet which is something like this 1,20,343,234,234,234,543,543,543,343
		C = textscan(s, '%d%d%d%d%d%d%d%d%d%d','delimiter',',');
		A(i,:) = cell2mat(C);
		%I plot just from the 3 value up to 10 th value (1 to 8 th sensor)
		for sID = 1:8
			PlotHandles(sID).YData = A(:,i+2);
		end
		
		drawnow;
	end
	n = n+1;
	flushinput(a);
end
hold(app.UIAxes,'off');
msgbox('Proceso Terminado');
xlswrite('OUTPUT',A);
end

[J. Alex Lee]

Triggering your save+display routine with the event supplied by the serial() object seems like a good idea, but it's not clear to me what you're asking with this new scheme.

I would still think you want to trigger only data saving when data is available. You can have that data saving function trigger another event to start the display function (which hopefully my above comment can help with), and apply some interruption logic to that so you can skip displays or whatever.

If you anticipate really going into the tens of thousands of data points, you might consider resampling the data for the display to ensure that you never have more than some number of data points displayed in the screen.

[Carlos Molina]

Thanks a lot for your answer, that was really helpfull.

I did some modification, I'm using now a animated line rather to plot and I update the X and Y data. Time of operation is quite good, about 0.01-0,04 seconds.

From the device I send the data as ASCI, but I will change after to send as a binari to save bytes and add check sum, etc. The device read the 8 sensors chanels more the power and send a colection of ASCI characters with NºSampling,Power,S1,....S8. After 8 packets of data I send an a LR terminator. In the matlab function, I read data when the LR character is received. Then I save all the data and I display just one of these 8 data packet, which is quite enough. Each data packet is sent from the device every 450 ms, so I have 8x450ms to perform everything in matlab.

Also, I make a security copy in the excel file where I want to store the data for further processing, so in case a crash happends, data lost from the last security copy is insignificant.

Well, rather to use the axes of the app, I use the tradicional figure, I prefer the format. Still I have some problems, for example, first reading get always damage, is not important because first data packet does not contains any important data, but I will figure out. Another issue is that sometime, after several thousend of data ocurrs a corruption in the data read or the data get lost. This could be just an error in the serial data communication, I will try to improve the serial protocol. Time of perform the reading keeps stable along the time so it does not relevant for the error, I think.

For the animated line, I set as maximum points 200 M points, which is more than I expect.

function  Leersensoractivo(com,fullfileName)


%Delete residual port
delete(instrfindall);
%Define serial port.
s = serial(com,'BaudRate',115200); 
%Set size of buffer
s.InputBufferSize = 1024;
if (s.Status == 'closed')       
    
    %fopenport is a own function as fopen but returns 0 or 1 if port can be open.
    st = fopenport(s);
    if st == 0
        errordlg('Port COM not available','Port COM error')
    else
        fopenport(s);
        figure('Name','ENOSE-DATA','NumberTitle','off');
        fig = gcf;
        grid on;
        %Creat an animated line for the 8 sensor and power.
        h1 = animatedline;
        h2 = animatedline;
        h3 = animatedline;
        h4 = animatedline;
        h5 = animatedline;
        h6 = animatedline;
        h7 = animatedline;
        h8 = animatedline;
        pw = animatedline;
        set(h1,'Color','b','MaximumNumPoints',200000000,'DisplayName','s1');
        set(h2,'Color','r','MaximumNumPoints',200000000,'DisplayName','s2');
        set(h3,'Color','g','MaximumNumPoints',200000000,'DisplayName','s3');
        set(h4,'Color','m','MaximumNumPoints',200000000,'DisplayName','s4');
        set(h5,'Color',[0.8500 0.5000 0.1000],'MaximumNumPoints',200000000,'DisplayName','s5');
        set(h6,'Color',[0.8500 0.3250 0.0980],'MaximumNumPoints',200000000,'DisplayName','s6');
        set(h7,'Color',[0.4940 0.1840 0.5560],'MaximumNumPoints',200000000,'DisplayName','s7');
        set(h8,'Color',[0.4660 0.6740 0.1880],'MaximumNumPoints',200000000,'DisplayName','s8');
        set(pw,'Color','k','MaximumNumPoints',200000000,'DisplayName','Power','LineWidth',0.8);
        
        
        global A;
        A = zeros(1,11);
        %Since the first xlswrite execution is the slowest, I perform out before start to read serial port.
        xlswrite(fullfileName,A);
        A = zeros(200,11);
        global i;
        global contador;
        global numitr;
        numitr = 1;
        contador = 0;
        i=1;
        s.BytesAvailableFcn = {@Call,fig,h1,h2,h3,h4,h5,h6,h7,h8,pw,fullfileName} ;
    end
end

end

function Call(obj,event,fig,h1,h2,h3,h4,h5,h6,h7,h8,pw,fullfileName)
tic; 
global contador;
global numitr;
global i;
global A;
M = zeros(1,10);


if ishghandle(fig)==1


      cadena = fscanf(obj);
      N = textscan(cadena, '%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d%d','delimiter',',');
      C = cell2mat(N);
    
      for k=1:10
          M(k) = double(C(k));%I take the firs data packet of the 8 to display.
      end
      b = numitr;
      addpoints(pw,b,M(2));
      addpoints(h1,b,M(3));
      addpoints(h2,b,M(4));
      addpoints(h3,b,M(5));
      addpoints(h4,b,M(6));
      addpoints(h5,b,M(7));
      addpoints(h6,b,M(8));
      addpoints(h7,b,M(9));
      addpoints(h8,b,M(10));
      n = i;
      %Save data.
      A(n,1:10) = C(1:10);
     
      n=n+1;
      A(n,1:10) = C(11:20);
 
      n=n+1;
      A(n,1:10) = C(21:30);

      n=n+1;
      A(n,1:10) = C(31:40);

      n = n+1;
      A(n,1:10) = C(41:50);
      n = n+1;
      A(n,1:10) = C(51:60);
      n = n+1;
      A(n,1:10) = C(61:70);
      n = n+1;
      A(n,1:10) = C(71:80);

      
      
      numitr = numitr + 8;
      axis([0,numitr,0,1000]);
      drawnow update;
      n = n+1;
      i = n;
      if i == 47 %Here I carry out the secury copy. and reset I, so I'm doing a
          %security copy every 48 readings.
          
          p = contador*(i-1)+1;
          xlswrite(fullfileName,A(1:i-1,:),1,sprintf('A%d',p));
          contador = contador +1;
          
          clear A;
          global A;
          A = zeros(200,11);
          i=1;
      end
     A(n-1,11) = toc;%I store the tic-toc time to control time of execution.
else
    %If user close the figure, I store the remained data from the last security copy
    B = A(1:i-1,:);
    a = numitr;
    p = a-i+1;
    xlswrite(fullfileName,B,1,sprintf('A%d',p));
    msgbox('xls file created');
    fclose(obj);
   
end
end