Reading JSON struct from file adds prefix 'x' to fieldname if it starts with number

I am reading JSON file using readstruct, and it works well otherwise, but for fieldnames that would start with a number there is 'x' added to the beginning. I have tried to search info for this feature, both for Matlab and JSON standard in general, but I cannot find anything that would explain why 'x' is added.

1. Is this feature only in Matlab, or is it something defined for JSON standard ?
2. Is there a way to disable this prefix or change it to something else?

Note that I cannot affect to the JSON files how they are originally written.

[Stephen23]

https://www.mathworks.com/help/matlab/ref/struct.html

"Field names can contain ASCII letters (A–Z, a–z), digits (0–9), and underscores, and must begin with a letter. The maximum length of a field name is namelengthmax."

Answer 1

[Matt J]

but I cannot find anything that would explain why 'x' is added.

Presumably you know that something must be added, because variable and field names starting with numbers are illegal in Matlab.

>> S.3abc=5

S.3abc=5

↑

Invalid expression. Check for missing multiplication operator, missing or unbalanced delimiters, or other syntax

error. To construct matrices, use brackets instead of parentheses.

If you want a different prefix, one option would be to post-process the field names, e.g.,

S.a=2;
S.b=3;
S.x1=7;
S.x2=11

S = struct with fields:
     a: 2
     b: 3
    x1: 7
    x2: 11

prefix="y";

F=string(fieldnames(S))';

idx=startsWith(F,"x"+digitsPattern);

F=F(idx);

for k=1:numel(F)

        f=F{k};
        fnew=prefix+f(2:end);
        S.(fnew)=S.(f);
    
end

S=rmfield(S,F)

S = struct with fields:
     a: 2
     b: 3
    y1: 7
    y2: 11

[Petri M]

Funny thing is, I have been using Matlab for 30 years but never happened to bump into the restriction that struct field names cannot start with a number :) If you think it, there should be no reason for the restriction: I understand the struct name like any other variable should start with a letter, but for field names I cannot think any example where using a number in the beginning could be confused with something else than a field name. Like for example 1i as such means imaginary value but a.1i could be explicitly identified as struct.

Another thing, it just so happens that JSON file has already some field names like x2, x4 etc., so it makes the disctinction really difficult. The numbers after x are power of two and field names starting with a number are not, but relying on that is a hazard. So if there were a way to define prefix that would be great.

But anyway, thanks for the explanation !

[Steven Lord]

The rule is that field names, variable names, and function names have to obey basically the same rules. They generally have to satisfy the isvarname function (the one rule that isvarname checks that gets a bit fuzzy is the keyword rule, as you can define an end method for a class despite it being a keyword, and there are functions like methods and properties that are keywords when used in classdef files.)

Fields in a struct array can become variables using save with the -struct option and load without an output argument.

cd(tempdir)
S = struct('x', 1, 'y', 2);
whos

  Name      Size            Bytes  Class     Attributes

  S         1x1               290  struct              

Neither x nor y are variables in the workspace at this point.

save('temporaryMatfile.mat', '-struct', 'S')
whos -file temporaryMatfile.mat

  Name      Size            Bytes  Class     Attributes

  x         1x1                 8  double              
  y         1x1                 8  double              

Note that this MAT-file contains variables x and y.

clear all
whos

There's nothing in the workspace so there's no output from whos.

load temporaryMatfile.mat
whos

  Name      Size            Bytes  Class     Attributes

  x         1x1                 8  double              
  y         1x1                 8  double              

Now x and y are variables in the workspace.

If structs could have fields whose names started with numbers, you could have variables whose names start with numbers. And we will not allow you to define, for example:

% S = struct('1', 2, '2', 1)
% save and load the struct to create variables in the workspace
% y = 1 + 1 % Would this result in y being 4???

That would be an OMDB "feature".

[Walter Roberson]

@Steven Lord

fieldnames for tables() can start with numbers, though.

T = table([3;5], 'VariableNames', "3141")

T = 2×1 table
    3141
    ____

     3  
     5  

The difference is that you cannot load() the variables of a table directly as variables.

[Petri M]

@Steven Lord Okok, thank you for giving the reasoning behind the rule.

Although, your last example is a bit extreme: I was not talking about field names being only numbers but just starting with a number. So, even if we could have a struct like S = struct('1a', 2, '2a', 1) and we could save and load the struct, there would be no danger to ever confuse variables 1a and 2a with numerals. Only exception would be imaginary numbers, so a number + 'i' (or 'j') would be a problem. Well, 'e' is also a problem if there are further numbers after e. But anyway I got the point already: fieldnames must obey the same rules as variables because it's possible to save struct fields as independent variables.

[Steven Lord]

You'd need to avoid #d..., #e..., #i, and #j.

format shortg
x = [1d2; 1e2; 1i; 1j]

x = 
          100 +          0i
          100 +          0i
            0 +          1i
            0 +          1i

You'd also need to avoid 0x... and 0b....

y = [0xff; 0b111]

y = 2×1

255 7

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

There may be other number-letter leading combinations that you'd have to avoid, but it's late here and I can't think of them off the top of my head.

[Petri M]

@Steven Lord Sure, generality would be an issue. However, I cannot resist the temptation to point out the filosofical question of whether zero is actually a number ;) But please no, don't bother answering this. I know what the answer is in terms of mathematics.

Answer 2

[Catalytic]

Note that I cannot affect to the JSON files how they are originally written.

It is easy enough to write a function to modify them -

modifyJSON('music.json', 'newmusic.json', 'ggg')

type music.json

{

"1Ensemble": {

"2Music": "jazz",

"3BandName": "Kool Katz",

"4Instrumentation": [

{

"Type": "wind",

"Instrument": "trumpet",

},

{

"Type": "percussion",

"Instrument": "piano",

"Pianotype": "concert grand",

},

{

"Type": "percussion",

"Instrument": "drums",

"Drumkit": [

"bass drum",

"floor tom",

"snare drum",

"hi-hat",

"ride cymbal"

],

},

{

"Type": "string",

"Instrument": "bass",

"Basstype": "upright"

}

]

},

"5Musicians": [

{

"Role": "trumpeter",

"Name": "Miles"

},

{

"Role": "vocalist",

"Name": "Roger"

},

{

"Role": "pianist",

"Name": "Diana"

},

{

"Role": "drummer",

"Name": "George"

},

{

"Role": "bassist",

"Name": "John"

}

]

}

type newmusic.json

{

"ggg1Ensemble": {

"ggg2Music": "jazz",

"ggg3BandName": "Kool Katz",

"ggg4Instrumentation": [

{

"Type": "wind",

"Instrument": "trumpet",

},

{

"Type": "percussion",

"Instrument": "piano",

"Pianotype": "concert grand",

},

{

"Type": "percussion",

"Instrument": "drums",

"Drumkit": [

"bass drum",

"floor tom",

"snare drum",

"hi-hat",

"ride cymbal"

],

},

{

"Type": "string",

"Instrument": "bass",

"Basstype": "upright"

}

]

},

"ggg5Musicians": [

{

"Role": "trumpeter",

"Name": "Miles"

},

{

"Role": "vocalist",

"Name": "Roger"

},

{

"Role": "pianist",

"Name": "Diana"

},

{

"Role": "drummer",

"Name": "George"

},

{

"Role": "bassist",

"Name": "John"

}

]

}

function modifyJSON(oldfile,newfile, prefix)
arguments
 oldfile string, newfile string 
 prefix string
end

   Str=readlines(oldfile);

   pat=whitespacePattern+'"'+digitsPattern+alphanumericsPattern+'":';

   k=startsWith(Str,pat);


   str=Str(k);

  for i=1:numel(str)

      s=extractBetween(str(i),  whitespacePattern+'"',  '":');

      str(i)=replaceBetween(str(i),  whitespacePattern+'"',  '":', prefix+s);

  end

  Str(k)=str;

  writelines(Str, newfile);

end

[Matt J]

You could also use jsondecode to go directly to a struct if your original file are strictly JSON compliant (e.g., no trailing commas),

outputStruct = jsondecode(strjoin(Str,newline))