I'm having an issue for some time where rclone uploads the same file over and over to google drive. It creates copies of the files on the server, sometimes 20-40 copies.
I also caught my server spawning many rclone processes, sometimes a hundred.
I have no idea on how to go about error checking this as there is nothing in the logs.
I run it on linux Ubuntu 18.04, rclone v1.48.0.
My exact cronjob is:
/usr/bin/rclone copy /mnt/local/downloads/files/allfiles/transfer gdrive:Files/bmn -v --tpslimit 5 --transfers 2 >> /home/seed/logs/transferout.log 2>&1
I have tried to limit transfers as can be seen above as well as chaging the --tpslimit. No effect
Any help appreciated. Its driving me crazy.
Can you pick one file and run a copy command and upload with -vv on it and share the log?
Are you running rclone off a crontab? You need to make sure that one has stopped before the next one starts. A really easy way of doing this is with adding
--rc to the command, but there are more sophisticated ways!
I'll be sure to add -vv. Regretfully it doesn't happen all the time so I cannot easily replicate it. I'll paste the log when it happens again.
Thanks for chiming in!
Thanks for chiming in @ncw.
Yea i'm running it off crontab, but it only executes once every 24hrs.
I also had the problem when running off a script where I had several rclone jobs execute one after the other. I moved to crontab to see if it would help. The script also only ran once every 24 hrs.
How were you identifying hundreds running?
Well normally I have some 240'ish processes on my machine. When it gets crazy with rclone it exceeds 500 and when I do
pidof rclone it shows like alot.
Granted I haven't counted them all but it is alot, and the machine is also sluggish. I normally just kill them with
What was your crontab entry? Were you able to validate it was the actual cron command going off all those times?
The crontab entry is
* 03 * * * /usr/bin/rclone copy /mnt/local/downloads/files/allfiles/transfer gdrive:Files/bmn -v --tpslimit 5 --transfers 2 >> /home/seed/logs/transferout.log 2>&1
It works most of the time but then sometimes it just goes crazy.
I'll check the syslog to see if I can see multiple crontabs going off.
Thanks for coming up with suggestions @Animosity022.
That way that's written, you'd get one launched every minute at 3am which would explain the multiples.
So my upload runs at 3:12AM each night:
# Cloud Upload
12 3 * * * /opt/rclone/scripts/upload_cloud
Right so it looks like I'm idiot. Thanks @Animosity022.
That would also explain why I only see the error intermittently - I have fiddled often with the crontab often to see if that makes any difference and I have on occasion changed the minute to an absolute number, which I will do again.
Thanks for the fast fault checking, its appreaciated!
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.