Is there any equivalent to lag function for non-regular timetable?

I was using financial time series (fints) up to now, but it appears that they are going to be removed in a future release.

Then, as advised, I am trying to switch to timetable; however, my series aren't regular (the series have data only for working days). The function 'lagts' was working fine with non-regular series, but it's not the case for 'lag' function. Is there any way to apply lag on non-regular series without having to retime them?

Thanks.

Answer 1

[Peter Perkins]

I think that by, "without having to retime them", you mean retime to a regular time vector.

If what you want is to create a value at t_i that is the interpolated value at t_i - delta_t, then you can use retime to interpolate the original timetable at all the (t_i - delta_t) values, then simply add delta_t to the time vector of that result.

[Maxime Nguyen]

Hi Peter, Thanks for the answer.

As I asked, I wanted to know if there was any way to "lag" the series without having to retime the series, just like for financial time series (fints).

I figured that there isn't any built-in way to do it for now, so I wrote a function to achieve that.

Answer 2

[Rick]

Maxime,

In the event the data is regular, timetable/lag and fints/lagts do the same thing. However, all fints/lagts does is shift data "down rows" without any regard for the datestamps. In general, this is not a lagging operation in a conventional sense, and so timetable/lag fails (as it should).

That said, if all you want to do is shift rows, then a simple initial data padding (either with zeros as fints/lagts does, which I feel is conceptually incorrect and should be NaNs) and property assignment is all you need when working with timetable/lag.

Consider the following code, which "lags" time series data by 3 rows:

t = (datetime('1-Dec-2018'):caldays:datetime('31-Dec-2018'))';
rng(200)
Price = 100 + 0.1*(0:numel(t) - 1)'.*cumsum(randn(numel(t),1)/100); 
TT    = timetable(Price,'RowTimes',t); % This is a REGULAR timetable
L     = 3;                             % Number of "lags" (i.e., rows to shift)

%
% Create a FINTS and compare to TT: 
%
%   o These are REGULAR time series and so TIMETABLE/LAG and FINTS/LAGTS
%     are both lagging data and produce the same results
%

FTS = fints(datenum(TT.Time), TT.Price, 'Price');

FTS_Lag = lagts(FTS,L)
TT_Lag  = lag(TT,L)

%
% Now create IRREGULAR time series: 
%
%   o TIMETABLE/LAG will now error because TT is irregular.
%   o FINTS/LAGTS works but all it's doing is shifting the data down L rows
%     which is really not a lagging operation is a conventional sense.
%   o To mimic the same behavior as FINTS/LAGTS, simply pad the first L
%     rows with zeros (or NaNs) and strip off the last L rows of TT
%

FTS = FTS([1 2 4 5 6 8 9 12 13:20 23:27]);
TT  =  TT([1 2 4 5 6 8 9 12 13:20 23:27],:);

FTS      = lagts(FTS, L)
TT.Price = [zeros(L,1) ; TT.Price(1:end-L)]