Get Started with MDF Datastore
This example shows you how to use the MDF datastore feature of Vehicle Network Toolbox™ to quickly and efficiently process a data set spread across a collection of multiple MDF files. This workflow is also valuable when there are too much data to fit into available memory.
Access MDF Files in a Datastore
Find the collection of MDF files representing logged information from multiple test sequences. Note that MDF files to be used by MDF datastore as a set must have the same channel group and channel content structure.
dir("File*.mf4")
File01.mf4 File02.mf4 File03.mf4 File04.mf4 File05.mf4
Create an MDF Datastore
You create an MDF datastore by selecting a folder location containing a collection of MDF files. In this case, target all files in the current working directory.
mds = mdfDatastore(pwd)
mds = MDFDatastore with properties: Datastore Details Files: { '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File01.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File02.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File03.mf4' ... and 2 more } ChannelGroups: GroupNumber AcquisitionName Comment ... and 10 more columns ___________ _______________ _____________ 1 <undefined> Integer Types {0x0 double} 2 <undefined> Float Types {0x0 double} Channels: Name GroupNumber DisplayName ... and 17 more columns _________________ ___________ ___________ "Float_32_LE" 2 "" {0x0 double} "Float_64_LE" 2 "" {0x0 double} "Signed_Int16_LE" 1 "" {0x0 double} ... and 3 more rows Options SelectedChannelNames: { 'Signed_Int16_LE'; 'Unsigned_UInt32_LE'; 'time' } SelectedChannelGroupNumber: 1 ReadSize: "file" ReadRaw: 0 ValidityRule: "include" IncludeEvents: 1
Configure MDF Datastore
Multiple options allow control of what data are read from the MDF files and how the reads are performed. In this case, the first channel group is used by default. Note that only one channel group may be selected by the datastore at a time. You can also specify certain channels within the selected channel group to read. In this case, all channels are read by default.
mds.SelectedChannelGroupNumber
ans = 1
mds.SelectedChannelNames
ans = 3x1 string
"Signed_Int16_LE"
"Unsigned_UInt32_LE"
"time"
Preview MDF Datastore
Using the preview
function, you can obtain a quick view of the data available in the file set. Preview always returns up to eight data points from the first file in the datastore.
preview(mds)
ans=8×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
5 sec 5 5
6 sec 6 6
7 sec 7 7
Read All Data in MDF Datastore
You can use the readall
function to read the entire data in a single call. This is an efficient way to read from many files when the data set fits into available memory. After running readall
, the datastore resets to the beginning of the data set.
data = readall(mds); data(1:5,:)
ans=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
Read a Subset of Data in MDF Datastore
You can use the read
function to obtain a subset of data from the datastore. The size of the subset is determined by the ReadSize
property of the MDF datastore object. By default, data from an entire file are read in one call. The power of a datastore comes from reading through multiple files sequentially within the file set. As you read, the datastore automatically bridges from one file to the next until all data from all files are read.
for ii = 1:3 data = read(mds); whos("data") data(1:5,:) end
Name Size Bytes Class Attributes data 10000x2 246364 timetable
ans=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
Name Size Bytes Class Attributes data 10000x2 246364 timetable
ans=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
Name Size Bytes Class Attributes data 10000x2 246364 timetable
ans=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
Reset MDF Datastore
At any time, you can call the reset
function to start over at the beginning of the data set.
reset(mds)
Configure Number of Records to Read from MDF Datastore
You can use the ReadSize
property to specify how much data to read on each call. ReadSize
can be specified as a numeric value to read a fixed number of data points. ReadSize
lets you control how much data is loaded into memory when you have a data set larger than available memory. It is recommended to use custom read sizes that are small enough to fit in memory, but still as large as possible to reduce processing overhead and improve performance.
mds.ReadSize = 5
mds = MDFDatastore with properties: Datastore Details Files: { '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File01.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File02.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File03.mf4' ... and 2 more } ChannelGroups: GroupNumber AcquisitionName Comment ... and 10 more columns ___________ _______________ _____________ 1 <undefined> Integer Types {0x0 double} 2 <undefined> Float Types {0x0 double} Channels: Name GroupNumber DisplayName ... and 17 more columns _________________ ___________ ___________ "Float_32_LE" 2 "" {0x0 double} "Float_64_LE" 2 "" {0x0 double} "Signed_Int16_LE" 1 "" {0x0 double} ... and 3 more rows Options SelectedChannelNames: { 'Signed_Int16_LE'; 'Unsigned_UInt32_LE'; 'time' } SelectedChannelGroupNumber: 1 ReadSize: 5 ReadRaw: 0 ValidityRule: "include" IncludeEvents: 1
for ii = 1:3 data = read(mds) end
data=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
data=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
5 sec 5 5
6 sec 6 6
7 sec 7 7
8 sec 8 8
9 sec 9 9
data=5×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
______ _______________ __________________
10 sec 10 10
11 sec 11 11
12 sec 12 12
13 sec 13 13
14 sec 14 14
Configure a Time Range to Read from MDF Datastore
You can also specify ReadSize
as a duration to read data points by elapsed time. Note that when the read type is changed, the datastore resets to the beginning of the data set.
mds.ReadSize = seconds(5)
mds = MDFDatastore with properties: Datastore Details Files: { '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File01.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File02.mf4'; '/tmp/Bdoc24b_2679053_3933560/tpa5e3c087/vnt-ex10761765/File03.mf4' ... and 2 more } ChannelGroups: GroupNumber AcquisitionName Comment ... and 10 more columns ___________ _______________ _____________ 1 <undefined> Integer Types {0x0 double} 2 <undefined> Float Types {0x0 double} Channels: Name GroupNumber DisplayName ... and 17 more columns _________________ ___________ ___________ "Float_32_LE" 2 "" {0x0 double} "Float_64_LE" 2 "" {0x0 double} "Signed_Int16_LE" 1 "" {0x0 double} ... and 3 more rows Options SelectedChannelNames: { 'Signed_Int16_LE'; 'Unsigned_UInt32_LE'; 'time' } SelectedChannelGroupNumber: 1 ReadSize: 5 sec ReadRaw: 0 ValidityRule: "include" IncludeEvents: 1
for ii = 1:3 data = read(mds) end
data=6×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
_____ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
5 sec 5 5
data=11×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
______ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
5 sec 5 5
6 sec 6 6
7 sec 7 7
8 sec 8 8
9 sec 9 9
10 sec 10 10
data=16×2 timetable
time Signed_Int16_LE Unsigned_UInt32_LE
______ _______________ __________________
0 sec 0 0
1 sec 1 1
2 sec 2 2
3 sec 3 3
4 sec 4 4
5 sec 5 5
6 sec 6 6
7 sec 7 7
8 sec 8 8
9 sec 9 9
10 sec 10 10
11 sec 11 11
12 sec 12 12
13 sec 13 13
14 sec 14 14
15 sec 15 15
Close MDF Files
Close access to the MDF files by clearing the MDF datastore variable from workspace.
clear mds