i don't know why these files are gone lol but anyway do you think this is an issue that make it take all of this time?
I wouldn't be able to tell you why they are gone as I don't think that's related.
As I noted above, you'd have to run with debug and share a log.
i don't how to do it
should i add in my cronjob command "-vv" "--log-file" "mylogfile.txt
Yes, that's it. I'd probably put the logfile in a spot you know where it will be so --log-file /tmp/rclone.log
or wherever you want.
Does this help with anything??
https://gofile.io/d/42eSXw
please download file as it exceeded the maximum upload
What's your upload speed? That log looks good with no errors.
upsteam 260 Gbit/s
Port / bandwidth 1000 Mbit/s port
UNLIMITED Traffic
You'd want to increase transfers and checkers to increase speeds. Try adding until you can figure out what your server can handle.
if i increased the transfer from 4 to 16 it will freeze my server while transfering
Not sure why you'd jump from 4 to 16. Go 6, that works, go 8, etc.
If you server isn't stable or capable, you live with the speeds. IF you have capacity, increase the transfers until you find the right number for your configuration.
1 Like
You will transfer more files but every file will take longer to upload.
If you want to increase speeds, --drive-chunk-size=128M or --drive-chunk-size=256M will increase your upload speeds. You will need the RAM per file, so keep that in mind.
You could be hitting a hardware bottleneck, most likely the IO. If you're using a hard drive, you'll never hit the speeds you want.
FYI, with 256M chunk, 4 transfers on a GCE VM using the standard persistent SSD storage (not even adding an NVME raid), I was able to hit 400MB/s sync directly between 2 google drive remotes, so it didn't even touch the ssd disk.
On an 1.5Tb NVME raid, uploading from nvme to google, I hit about 200MB/s with > 8GB files.
Install netdata and you'll likely see high iowait and stall pressure.
You are thinking Google Drive, but he's using Backblaze which benefits a lot from more transfers. I definitely agree on the Google Drive statements 
so i should do what he is doing as it will be better only with Google Drive?
Those parameters he shared only work with Google Drive.
mylog.txt (1.3 MB)
ok so i made the transfer at 8 and it took 8:30 mins to transfer please check the log file. Thank you
No need to run in a debug unless you are getting errors.
Just tweak and find what runs best for your setup.
mylog.txt (1.3 MB)
i have done it with transfer 4 only. Thankfully it didn't freeze the server but it took almost 10 hours to transfer which i think it's really too much time. is there any solution to speed the transfer ?
We are circling here.
You need to either:
- fix the server stability and increase transfers
- live with the speed with 4
increase to 6/8/10/12/14/16/etc and see the breaking point.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.