Retime and Synchronize Timetable Variables Using Different Methods

Open Live Script

This example shows how to fill in gaps in timetable variables, using different methods for different variables. You can specify whether each timetable variable contains continuous or discrete data using the VariableContinuity property of the timetable. When you resample the timetable using the retime function, retime either interpolates, fills in with previous values, or fills in with missing data indicators, depending on the values in the VariableContinuity property. Similarly, the synchronize function interpolates or fills in values based on the VariableContinuity property of the input timetables.

Create Timetable

Create a timetable that has simulated weather measurements for several days in May 2017. The timetable variables Tmax and Tmin contain maximum and minimum temperature readings for each day, and PrecipTotal contains total precipitation for the day. WXEvent is a categorical array, recording whether certain kinds of weather events, such as thunder or hail, happened on any given day. The timetable has simulated data from May 4 to May 10, 2017, but is missing data for two days, May 6th and May 7th.

Date = [datetime(2017,5,4) datetime(2017,5,5) datetime(2017,5,8:10)]';
Tmax = [60 62 56 59 60]';
Tmin = [44 45 40 42 45]';
PrecipTotal = [0.2 0 0 0.15 0]';
WXEvent = [2 0 0 1 0]';
WXEvent = categorical(WXEvent,[0 1 2 3 4],{'None','Thunder','Hail','Fog','Tornado'});
Station1 = timetable(Date,Tmax,Tmin,PrecipTotal,WXEvent)
Station1=5×4 timetable
       Date        Tmax    Tmin    PrecipTotal    WXEvent
    ___________    ____    ____    ___________    _______

    04-May-2017     60      44         0.2        Hail   
    05-May-2017     62      45           0        None   
    08-May-2017     56      40           0        None   
    09-May-2017     59      42        0.15        Thunder
    10-May-2017     60      45           0        None

Resample Continuous and Discrete Timetable Variables

One way to fill in data for the two missing days is to use the retime function. If you call retime without specifying a method, then retime fills in gaps with missing data indicators. For instance, retime fills gaps in numeric variables with NaN values, and gaps in the categorical variable with undefined elements.

Station1Daily = retime(Station1,'daily')
Station1Daily=7×4 timetable
       Date        Tmax    Tmin    PrecipTotal      WXEvent  
    ___________    ____    ____    ___________    ___________

    04-May-2017     60      44         0.2        Hail       
    05-May-2017     62      45           0        None       
    06-May-2017    NaN     NaN         NaN        <undefined>
    07-May-2017    NaN     NaN         NaN        <undefined>
    08-May-2017     56      40           0        None       
    09-May-2017     59      42        0.15        Thunder    
    10-May-2017     60      45           0        None

If you specify a method when you call retime, it uses the same method to fill gaps in every variable. To apply different methods to different variables, you can call retime multiple times, each time indexing into the timetable to access a different subset of variables.

However, you also can apply different methods by specifying the VariableContinuity property of the timetable. You can specify whether each variable contains continuous or discrete data. Then the retime function applies a different method to each timetable variable, depending on the corresponding VariableContinuity value.

If you specify VariableContinuity, then the retime function fills in the output timetable variables using the following methods:

  • 'unset' — Fill in values using the missing data indicator for that type (such as NaN for numeric variables).

  • 'continuous' — Fill in values using linear interpolation.

  • 'step' — Fill in values using previous value.

  • 'event' — Fill in values using the missing data indicator for that type.

Specify that the temperature data in Station1 is continuous, that PrecipTotal is step data, and that WXEvent is event data.

Station1.Properties.VariableContinuity = {'continuous','continuous','step','event'};
Station1.Properties
ans = 
  TimetableProperties with properties:

             Description: ''
                UserData: []
          DimensionNames: {'Date'  'Variables'}
           VariableNames: {'Tmax'  'Tmin'  'PrecipTotal'  'WXEvent'}
           VariableTypes: ["double"    "double"    "double"    "categorical"]
    VariableDescriptions: {}
           VariableUnits: {}
      VariableContinuity: [continuous    continuous    step    event]
                RowTimes: [5×1 datetime]
               StartTime: 04-May-2017
              SampleRate: NaN
                TimeStep: NaN
                  Events: []
        CustomProperties: No custom properties are set.
      Use addprop and rmprop to modify CustomProperties.

Resample the data in Station1. Given the values assigned to VariableContinuity, the retime function interpolates the temperature data, fills in the previous day's values in PrecipTotal, and fills in WXEvent with undefined elements.

Station1Daily = retime(Station1,'daily')
Station1Daily=7×4 timetable
       Date        Tmax     Tmin     PrecipTotal      WXEvent  
    ___________    ____    ______    ___________    ___________

    04-May-2017     60         44        0.2        Hail       
    05-May-2017     62         45          0        None       
    06-May-2017     60     43.333          0        <undefined>
    07-May-2017     58     41.667          0        <undefined>
    08-May-2017     56         40          0        None       
    09-May-2017     59         42       0.15        Thunder    
    10-May-2017     60         45          0        None

If you specify a method, then retime applies that method to all variables, overriding the values in VariableContinuity.

Station1Missing = retime(Station1,'daily','fillwithmissing')
Station1Missing=7×4 timetable
       Date        Tmax    Tmin    PrecipTotal      WXEvent  
    ___________    ____    ____    ___________    ___________

    04-May-2017     60      44         0.2        Hail       
    05-May-2017     62      45           0        None       
    06-May-2017    NaN     NaN         NaN        <undefined>
    07-May-2017    NaN     NaN         NaN        <undefined>
    08-May-2017     56      40           0        None       
    09-May-2017     59      42        0.15        Thunder    
    10-May-2017     60      45           0        None

Synchronize Timetables That Contain Continuous and Discrete Data

The synchronize function also fills in output timetable variables using different methods, depending on the values specified in the VariableContinuity property of each input timetable.

Create a second timetable that contains pressure readings in millibars from a second weather station. The timetable has simulated readings from May 4 to May 8, 2017.

Date = datetime(2017,5,4:8)';
Pressure = [995 1003 1013 1018 1006]';
Station2 = timetable(Date,Pressure)
Station2=5×1 timetable
       Date        Pressure
    ___________    ________

    04-May-2017       995  
    05-May-2017      1003  
    06-May-2017      1013  
    07-May-2017      1018  
    08-May-2017      1006

Synchronize the data from the two stations using the synchronize function. synchronize fills in values for variables from Station1 according to the values in the VariableContinuity property of Station1. However, since the VariableContinuity property of Station2 is empty, synchronize fills in Pressure with NaN values.

BothStations = synchronize(Station1,Station2)
BothStations=7×5 timetable
       Date        Tmax     Tmin     PrecipTotal      WXEvent      Pressure
    ___________    ____    ______    ___________    ___________    ________

    04-May-2017     60         44        0.2        Hail              995  
    05-May-2017     62         45          0        None             1003  
    06-May-2017     60     43.333          0        <undefined>      1013  
    07-May-2017     58     41.667          0        <undefined>      1018  
    08-May-2017     56         40          0        None             1006  
    09-May-2017     59         42       0.15        Thunder           NaN  
    10-May-2017     60         45          0        None              NaN

To indicate that Station2.Pressure contains continuous data, specify the VariableContinuity property of Station2. Though Station2 contains only one variable, you must specify VariableContinuity using a cell array, not a character vector.

Station2.Properties.VariableContinuity = {'continuous'};
Station2.Properties
ans = 
  TimetableProperties with properties:

             Description: ''
                UserData: []
          DimensionNames: {'Date'  'Variables'}
           VariableNames: {'Pressure'}
           VariableTypes: "double"
    VariableDescriptions: {}
           VariableUnits: {}
      VariableContinuity: continuous
                RowTimes: [5×1 datetime]
               StartTime: 04-May-2017
              SampleRate: NaN
                TimeStep: 1d
                  Events: []
        CustomProperties: No custom properties are set.
      Use addprop and rmprop to modify CustomProperties.

Synchronize the data from the two stations. synchronize fills in values in BothStations.Pressure because Station2.Pressure has continuous data.

BothStations = synchronize(Station1,Station2)
BothStations=7×5 timetable
       Date        Tmax     Tmin     PrecipTotal      WXEvent      Pressure
    ___________    ____    ______    ___________    ___________    ________

    04-May-2017     60         44        0.2        Hail              995  
    05-May-2017     62         45          0        None             1003  
    06-May-2017     60     43.333          0        <undefined>      1013  
    07-May-2017     58     41.667          0        <undefined>      1018  
    08-May-2017     56         40          0        None             1006  
    09-May-2017     59         42       0.15        Thunder           994  
    10-May-2017     60         45          0        None              982

If you specify a method as an input argument to synchronize, then synchronize applies that method to all variables, just as the retime function does.

See Also

| |

Topics