Using matlab with data on an external hard drive
50 views (last 30 days)
Show older comments
Hi Everyone,
I'm trying to run some data analysis through matlab. My data is quite large, and I am currently off site (so the VPN is extremely slow). I downloaded my data onto an external hard drive (Porsche Design - sold by apple) and am trying to run my analyses by accessing the portable hard drive. However, this has substantially slowed down my Matlab and it's extremely laggy.
Would any of you experts have a better way of doing this for a home office set up? Do I need to invest in a different data storage solution? My macbook pro has a 3.5 GHz Intel Core i7 processor with 16 GB of RAM, so I doubt this is the problem. My data set is huge, otherwise I would run it locally.
Any help/advice is much appreciated as always!
PK
0 Comments
Answers (4)
Walter Roberson
on 21 Aug 2018
I had one of those Porsche design external drives. It failed after a relatively short time, a number of years ago.
Looking at the remains, it looks like internally it was a 500 Gb Seagate Barracuda, 7200 rpm (which is a good thing), but that the interface was USB 2 or original Firewire (later called Firewire 200), both of which are rather slow for today's use.
Your 3.5 GHz MacBook Pro would have been produced after they stopped putting original Firewire into the MacBook Pro, so unless you used the Firewire convertor cable, you would be stuck using USB 2.0 with it.
Your 3.5 GHz MacBook Pro has USB 3.0 interfaces. I know that without bothering to look in my reference material about exactly which model and year yours is, because my MacBook Pro has USB 3.0 and it is older than yours is (I can tell by the clock speed.)
For mass storage, I am currently using a Seagate Backup Plus 5 TB, which benchmarks faster (about 150 megabits/s) than my internal Apple drive (88-ish megabits/s) on the iMac I am using. When I was doing the research, the Backup Plus was not (at the time) the fastest drive, but it was (at the time) one of the faster and more reliable, and the faster drives were (at the time) much more expensive or much less reliable.
0 Comments
PAK
on 23 Aug 2018
1 Comment
Walter Roberson
on 23 Aug 2018
I do not have any experience with SSD devices.
My reading of specifications is that you must have a 13 inch MacBook Pro from 2017 with the four Thunderbolt 3 ports and 7567U processor (i7 3.5 GHz). As that processor was an upgrade to the top-end model, I suspect that means you have 512 GB of SSD storage already. If so then you would find any non-SSD hard drive to be much much slower than what you have internally.
Larry B
on 24 May 2019
I had some very, very large data files (Got the size limitation alert) that I wanted to access from a external drive named "My Passport" in matlab using my MACBOOK PRO. The following is example of the path layout for the filename to make that happen:
fname ='/Volumes/My Passport/Tower_Collect/201808B.raw';
The large data set was successfully loaded and processed
1 Comment
Walter Roberson
on 24 May 2019
Yes, I have a Western Digital WorldBook, which is a NAS (Network Attached Storage) closely related to their Passport products. When I was researching before, the Passport series was not one of the faster options, and it was not notably less expensive either. Perhaps that has changed in more recent incarnations.
Hmmm, I see the 2.0 TB drive referred to as "fast" but the quality stories in the comments leave something to be desired; https://www.bhphotovideo.com/explora/computers/hands-review/review-western-digital-2tb-my-passport-portable-hard-drive
The 4.0 TB drive has been benchmarked as inexpensive but not quite as fast as alternatives especially for large files; https://www.pcworld.com/article/3178629/wd-my-passport-4tb-review-a-good-budget-choice-thats-just-a-little-slow.html
D.
on 21 May 2024
Syms x y z f=2*x*y-y*z; grad_f=(gradient(f),{x,y,z},{1,2,3}; display('grad(f)='); disp(grad_f')
0 Comments
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!