I ended up writing a custom Datastore class. In case others face the same problem, I recommend reading these pages carefully:
In my case, when I initialise my custom class, I pass two arguments: the file names of my signals and the file names of the corresponding labels. I then create a filedatastore for each dataset. What was a key observation is that your custom read function must return 1 observation at a time by default (this can be increased by implementing the ReadSize property). In this case that means then if I call ds.preview it will return a row that contains a 1x1024x2 signal and the corresponding categorical array.