Using matlab with data on an external hard drive

Hi Everyone,

I'm trying to run some data analysis through matlab. My data is quite large, and I am currently off site (so the VPN is extremely slow). I downloaded my data onto an external hard drive (Porsche Design - sold by apple) and am trying to run my analyses by accessing the portable hard drive. However, this has substantially slowed down my Matlab and it's extremely laggy.

Would any of you experts have a better way of doing this for a home office set up? Do I need to invest in a different data storage solution? My macbook pro has a 3.5 GHz Intel Core i7 processor with 16 GB of RAM, so I doubt this is the problem. My data set is huge, otherwise I would run it locally.

Any help/advice is much appreciated as always!

PK

Answer 1

[Walter Roberson]

I had one of those Porsche design external drives. It failed after a relatively short time, a number of years ago.

Looking at the remains, it looks like internally it was a 500 Gb Seagate Barracuda, 7200 rpm (which is a good thing), but that the interface was USB 2 or original Firewire (later called Firewire 200), both of which are rather slow for today's use.

Your 3.5 GHz MacBook Pro would have been produced after they stopped putting original Firewire into the MacBook Pro, so unless you used the Firewire convertor cable, you would be stuck using USB 2.0 with it.

Your 3.5 GHz MacBook Pro has USB 3.0 interfaces. I know that without bothering to look in my reference material about exactly which model and year yours is, because my MacBook Pro has USB 3.0 and it is older than yours is (I can tell by the clock speed.)

For mass storage, I am currently using a Seagate Backup Plus 5 TB, which benchmarks faster (about 150 megabits/s) than my internal Apple drive (88-ish megabits/s) on the iMac I am using. When I was doing the research, the Backup Plus was not (at the time) the fastest drive, but it was (at the time) one of the faster and more reliable, and the faster drives were (at the time) much more expensive or much less reliable.

Answer 2

[PAK]

Hi Walter,

Thanks for your response! I think that makes sense. I'm now looking into some of the USB-C SSD and HDD storage devices. Do you think getting a USB-SSD is worth the money? They are quite spendy!

PK

[Walter Roberson]

I do not have any experience with SSD devices.

My reading of specifications is that you must have a 13 inch MacBook Pro from 2017 with the four Thunderbolt 3 ports and 7567U processor (i7 3.5 GHz). As that processor was an upgrade to the top-end model, I suspect that means you have 512 GB of SSD storage already. If so then you would find any non-SSD hard drive to be much much slower than what you have internally.

https://www.macworld.co.uk/feature/mac/best-ssd-mac-3600870/

Answer 3

[Larry B]

I had some very, very large data files (Got the size limitation alert) that I wanted to access from a external drive named "My Passport" in matlab using my MACBOOK PRO. The following is example of the path layout for the filename to make that happen:

fname ='/Volumes/My Passport/Tower_Collect/201808B.raw';

The large data set was successfully loaded and processed

[Walter Roberson]

Yes, I have a Western Digital WorldBook, which is a NAS (Network Attached Storage) closely related to their Passport products. When I was researching before, the Passport series was not one of the faster options, and it was not notably less expensive either. Perhaps that has changed in more recent incarnations.

Hmmm, I see the 2.0 TB drive referred to as "fast" but the quality stories in the comments leave something to be desired; https://www.bhphotovideo.com/explora/computers/hands-review/review-western-digital-2tb-my-passport-portable-hard-drive

The 4.0 TB drive has been benchmarked as inexpensive but not quite as fast as alternatives especially for large files; https://www.pcworld.com/article/3178629/wd-my-passport-4tb-review-a-good-budget-choice-thats-just-a-little-slow.html

Answer 4

[D.]

Syms x y z f=2*x*y-y*z; grad_f=(gradient(f),{x,y,z},{1,2,3}; display('grad(f)='); disp(grad_f')