All - Activity - comodoro - DeepTalk - Deep Learning Community

Uploading large amounts of data

Technical Help
Jan 29, 2025
As I do not need it very often, I went with Backblaze, chunking the download into pieces and checking them afterwards. Speed is okay, it takes more time to unpack the data. Disadvantage is egress cost, for frequent uses of the data I would consider Wasabi.

H100 performance

Technical Help
May 24, 2023
I have tried the new H100 and have to admit my expectations were higher, which however may be caused by old code having bottlenecks other than GPU. But this is curious:
Epoch 0 | Training | Elapsed Time: 3:26:44 | Steps: 4013 | Loss: 58.975423
Epoch 0 | Validation | Elapsed Time: 0:03:45 | Steps:…

Best Practices(?): common tactics for server setups & data storage

Deep Learning: Getting Started
May 24, 2023
Hi,
I would think this very much depends on the circumstances. In my case, I have spent some time on a setup script, only to realize I would have to downgrade CUDA (I need 10.0). I have a Docker image that works out of the box, so I switched to that, accepting the small overhead.

Does processes end after ssh session disconnect?

Technical Help
May 11, 2023
I would add that nohup does just that (no termination on session end), simply instead of
$ command
use
$ nohup command

Uploading large amounts of data

Technical Help
Apr 12, 2023
All right, How to Transfer Data to Lambda Cloud GPU Instances claims that Lambda does not offer persistent storage (although this may be a bit out of date as there is a Storage item in my account). In that case, a question for everyone: what cloud solution do you use? Backblaze looks good, but I am …

Uploading large amounts of data

Technical Help
Apr 10, 2023
I need to use large amounts of data (~200GB) which cannot be that uncommon. What is the recommended method or dealing with this? Spinning up a GPU instance and copying into it is out of the question, I do not have a Gbps upload. So it has to be upload to some service and transferring it from there…