Rclone won't exit

Help and Support
1

What is the problem you are having with rclone?

Hi.. my rclone nightly jobs are suddenly NOT exiting..
i get this for hours, until i kill the proc from the command line..

2020/09/12 13:00:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:01:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

my script looks like this
#!/bin/bash

rclone drive

now=$(date +"%m%d_%Y")
file="/volume2/homes/rclonelog/logfile$_now.txt"
rclone -v -c sync /volume2/Rackstation rack:RackStation --exclude-from "/volume2/homes/exclusion.txt" --config="/var/services/homes/admin/.config/rclone/rclone.conf" --log-file=$_file --bwlimit "07:00,128 22:00,off"
egrep -w 'Copied|Deleted' $_file

i have no idea what has changed.. am i doing something stupid? thanks in advance. JP

2

edit -- i recently added the bandwidth restriction.. could that be it?

3

edit again -
rclone version 1.52.1 running on synology RS2418rp+ to google drive

5

What do the logs say?

6

2020/09/12 13:08:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:09:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:10:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:11:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:12:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:13:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:14:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:15:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:16:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:17:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

2020/09/12 13:18:50 INFO :
Transferred: 81.001M / 81.001 MBytes, 100%, 35.543 kBytes/s, ETA 0s
Checks: 84577 / 84577, 100%
Deleted: 1
Transferred: 15 / 15, 100%
Elapsed time: 38m53.6s

7

it will repeat this forever..
is there another log file i'm unaware of?
thanks for your reply
#JP

8

Try adding -vv to see the debug log.

Also try the latest rclone if you aren't using it already.

9

ah.. i see the problem.. having deleted a bunch of stuff, rmdir is going through the whole lot and trying to delete directories that are no longer on the source, but failing due to trashed files, and things like Thumbs.db and @eaDir files lying around..
is there a way to get rid of these pesky files?
rclone -vv -c deletefile rack:Thumbs.db --config="/var/services/homes/admin/.config/rclone/rclone.conf" --dry-run
didn't work

10

Can you post the log that that command generated so I can see the problem?

11 
                      2020/09/13 10:32:06 DEBUG : rclone: Version "v1.52.1" starting with parameters ["rclone" "-vv" "-c" "deletefile" "rack:Thumbs.db" "--config=/var/services/homes/admin/.config/rclone/rclone.conf" "--dry-run"]

2020/09/13 10:32:06 DEBUG : Using config file from "/var/services/homes/admin/.config/rclone/rclone.conf"
2020/09/13 10:32:14 ERROR : Attempt 1/3 failed with 1 errors and: rack:Thumbs.db is a directory or doesn't exist
2020/09/13 10:32:14 ERROR : Attempt 2/3 failed with 1 errors and: rack:Thumbs.db is a directory or doesn't exist
2020/09/13 10:32:14 ERROR : Attempt 3/3 failed with 1 errors and: rack:Thumbs.db is a directory or doesn't exist
2020/09/13 10:32:14 DEBUG : 4 go routines active
2020/09/13 10:32:14 Failed to deletefile: rack:Thumbs.db is a directory or doesn't exist

12

it doesn't exist in the top level.. but i was hoping i was hoping for a recursive delete.. there must be hundreds of thousands of thumbs.db hidden in there

13

You can use rclone purge for recursive delete. It works on directories not files.

Use -i for testing first!

14

i thought purge deleted everything! If i want to get fired I guess that might work. but i literally just want to get rid of empty directories, or rather directories with nothing in them but Thumbs.db. sync will do that for me once the directories are empty. however its not emptying them, even though they are no longer on the source, so i want to search through the entire google drive and delete any occurrences of Thumbs.db, rclone delete seems to search for a single directory in the root, and rclone deletefile looks for a file in the root only. rclone purge appears to delete the world out from under me.
going through google drive and deleting with the gui is impossible

15

I guess the real question is why does rclone sync leave these files intact, after i've deleted from the source?

16

I have scoured all remaining thumbs from the source, though of course as the directories in question aren't in existence locally it's not going to make a difference. However, watching the log during the latest sync, i notice that rclone sync is excluding my specified files from the sync that are present on the remote rather than locally.
2020/09/14 08:40:34 DEBUG : Departments/MEDICAL/.DS_Store: Excluded
2020/09/14 08:40:34 DEBUG : Departments/MEDICAL/Thumbs.db: Excluded
2020/09/14 08:40:34 DEBUG : Departments/BRIDGE/.DS_Store: Excluded
2020/09/14 08:40:34 DEBUG : Departments/BRIDGE/@eaDir: Excluded
2020/09/14 08:40:34 DEBUG : Departments/OFFICE/@eaDir: Excluded
2020/09/14 08:40:34 DEBUG : Departments/OFFICE/.DS_Store: Excluded
2020/09/14 08:40:34 DEBUG : Departments/DECK/.DS_Store: Excluded

this is confusing me.. i assumed exclusions were to prevent these files being uploaded to the remote destination, but they don't exist locally, so the exclusions must be checking the remote for these files.
this might explain why they aren't getting deleted.. but its baffling me

17

If you want things you excluded deleted then use --delete-excluded. This is another super dangerous flag so try first with -i!

  --delete-excluded             Delete files on dest excluded from sync
18

@Talitha

I am having a similar rclone not exiting problem as well (though I am not really seeing errors). It looks extremely similar to the issue you're having as well (though I am using fish shell). Is there anything about our environments that are similar? (I actually upgraded rclone manually but am still seeing the issue.).

(I seem not to be able to include links to posts in this forum, but it's titled as "Rclone not exiting on copy or sync (successful) to Dropbox crypt"

D.

D.

19

mine is exiting, but its taking a lot longer as theres so many empty dirs

20

that makes sense..
i went through the source with a fine tooth comb and deleted the last of the pesky thumbs.db, then i removed it from the exclusion list and lo and behold all the dirs with thumbs in were wiped..
i guess --delete-excluded does the same job.. I'll give it a try.
thanks.
meantime i';ll put thumbs back in the exclusion list, as i'll be generating more of course

1 Like
21

Good glad you got it working :slight_smile: