Cheapest way to upload data into the filesystem with slow upload bandwidth - Technical Help - DeepTalk - Deep Learning Community
Cheapest way to upload data into the filesystem with slow upload bandwidth
post by dushan on Jan 26, 2025
I just signed up and need to upload data into the lambdalabs filesystem before I can train on it. The docs state the only way to do this is by attaching the filesystem to an instance. But the only instances available are expensive GPUs.
I have 20Mbit/s upload speeds. Do I really need to launch a high powered GPU instance and run it for over 24hrs to get my data onto lambdalabs’ cloud?
Are there no ultra cheap instances that can be used for this?
I’m either going to have to pay to burn through GPU time, or upload to AWS or GCP and transfer from there. Seems like there must be a better way.
post by cody_b on Jan 31, 2025
@dushan Unfortunately, you’re correct that at this time, you need a GPU instance to use a filesystem.
It’s a bit painful, but you can upload your data somewhere (to Cloudflare R2, for example) then download your data from there directly to your instance.
post by banned4hax on Aug 21, 2025
If you really must, upload the file to a google drive folder you control, then make it accesible to anyone with the link, from there you should be able to simply perform a get request against it via the terminal.
post by utanapishtirox on Aug 26, 2025
And you could make it a password-protected archive file for security.