readtable not reading logical values as expected

48 views (last 30 days)
I am using version 9.9.0.1592791 (R2020b) Update 5.
I found this behavior while reading and writing tables via readtable() and writetable().
% First, I make a table and write it to file:
myint = 1;
mychar = {'1'};
mybool = true;
t = table();
t = addvars(t, myint);
t = addvars(t, mychar);
t = addvars(t, mybool)
writetable(t, 'mytable.csv');
% This gives the table t:
%{
t =
1×3 table
myint mychar mybool
_____ ______ ______
1 {'1'} true
%}
% and the .csv file:
%{
myint,mychar,mybool
1,1,1
%}
% Note that "true" is written as a "1" as is expected
% Now, I try to read the table, properly setting the options:
opts = detectImportOptions('mytable.csv');
opts = setvartype(opts,'myint','int8');
opts = setvartype(opts,'mychar','char');
opts = setvartype(opts,'mybool','logical');
t = readtable('mytable.csv',opts)
% Instead of the expected result of getting a table with values 1 (as an integer), "1" (as a char), and true (logical), I get:
%{
t =
1×3 table
myint mychar mybool
_____ ______ ______
1 {'1'} false
%}
% When I change my file to:
%{
myint,mychar,mybool
1,1,true
%}
% readtable() reads the data as expected, giving:
%{
t =
1×3 table
int char bool
___ _____ _____
1 {'1'} true
%}
Why is this happening? This is unexpected, especially considering that writetable writes logical values as '0' and '1'. Is this expected or is this a bug?
Note: When I use .xls instead of .csv, it behaves as expected.
  1 Comment
Adam Danz
Adam Danz on 10 Aug 2021
Edited: Adam Danz on 10 Aug 2021
Same behavior in R2021a. This is unexpected.
You should report this to tech support: Contact Us - MATLAB & Simulink
As a workaround, read in the logical column as double as then convert to logical.
opts = setvartype(opts,'mybool','double');
t = readtable('mytable.csv',opts);
t.mybool = logical(t.mybool);

Sign in to comment.

Accepted Answer

Jeremy Hughes
Jeremy Hughes on 10 Aug 2021
Edited: Jeremy Hughes on 10 Aug 2021
The default results of detectImportOptions seeing "1" & "0" will be numeric, and the default logical reading expects "true","t" or "false","f" (case insensitive), but this is overridable.
T = array2table(randn(6,3)>0)
T = 6×3 table
Var1 Var2 Var3 _____ _____ _____ true true true true true false true false true true true true false false true true false true
writetable(T,"mytable.csv","WriteVariableNames",false)
opts = delimitedTextImportOptions("NumVariables",3,...
"VariableNames",["myint","mychar","mybool"],...
"VariableTypes",["int8","char","logical"]);
getvaropts(opts,'mybool')
ans =
LogicalVariableImportOptions with properties: Variable Properties: Name: 'mybool' Type: 'logical' FillValue: 0 TreatAsMissing: {} QuoteRule: 'remove' Prefixes: {} Suffixes: {} EmptyFieldRule: 'missing' Logical Options: TrueSymbols: {'true' 't'} FalseSymbols: {'false' 'f'} CaseSensitive: 0
You can set what gets treated as true or false with the true and false symbols.
opts = setvaropts(opts,'mybool',...
"TrueSymbols",["t","true","1"],...
"FalseSymbols",["f","false","0"]);
t = readtable('mytable.csv',opts)
t = 6×3 table
myint mychar mybool _____ ______ ______ 1 {'1'} true 1 {'1'} false 1 {'0'} true 1 {'1'} true 0 {'0'} true 1 {'0'} true
Note: This will only match literally 0 and 1, not "0.0" as false or "2" true, but should be good enough for most cases.
  11 Comments
Jeremy Hughes
Jeremy Hughes on 21 Jun 2022
Hi, the reason for the quick closing of the case is that the report already exists in the system and it's not considered a bug. Your expectation and the design of this feature aren't in alignment, but it is working as designed as expected. This is, at least from a "policy" perspective, why it's an enhancement. Unfortunately, changing the default behavior would result in silent breaking-changes to existing uses which expect the current behavior. I realize that's not the most satisfying answer for your case.
Based on what you're expecting, instead of "logical" you'd be better off continuing to treat this as a number on import (uint8 perhaps), and convert it afterwards. There's not any performance ramifications for that, just a line:
t.mybool = logical(t.mybool)
That works as long as there aren't any "T" or "True" values as well. The paradigm of logical representation needs to be consistent in the file at the very least, or all hope is lost.

Sign in to comment.

More Answers (0)

Products


Release

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!