Rclone sync just stops at 504,333 checks

Help and Support
1

What is the problem you are having with rclone?

I've manually copied over a 383 GB directory with 651,557 files and am running rclone sync to check everything is there and make incremental changes. Even though nothing new needs to be copies, it's taken10+ hours to run 504,333 file checks then not done anything for the last few hours.

Run the command 'rclone version' and share the full output of the command.

- os/version: slackware 14.2+ (64 bit)
- os/kernel: 5.10.28-Unraid (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.3
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

smb via alias then crypt

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync /mnt/files crypt:/files --backup-dir=crypt:/.archive/files/2024/2024-01-08_11:36:22 --log-file=/mnt/rclone/rclone.log -P --modify-window=1s --filter-from=/mnt/rclone/filter_rules --transfers=8

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[backup]
type = alias
remote = /mnt/remotes/backup/files/

[crypt]
type = crypt
remote = backup:
password = XXX

A log from the command that you were trying to run with the -vv flag

Last entries in log are several hours old

2024/01/08 18:17:57 ERROR : personal-stuff/longfilename.jpg: Failed to copy: open /mnt/remotes/backup/files/2hc1kc2773q6o2gat2p4mji4lo/i8uj0rmsqs09kbahmasv2opduc/r5vsovl179v9olava4f3cm8bfc/45d14bojuf20956rfessofpdn57p238bhns8d17me1rftnenvot072antaf7tvi7jedk09o4vvb525ffrk8rkspbiesem2lmcbfaq3ipn93m1g1cle5npgugci58ff6af4s7q1r8r3go3605pe7qsckju3kf0624ohq67up0bt96hdj6de8cqn1vppk4n4ml4f7r93pmo3i0bqmq7cg92tvlc0r6d6tjg9tuvf01acci7rur604qj9k31n2lhh5miiddspto92udr5a8u56n3fu3lc8l6pivbscqolcragt7tciqb5jeml36dn4rdklns7lagbs27np58r78s0ju9pcpm9ovbusa1c7f3cbem01ddff7sdhjvgmqc5ln9i5p: file name too long
2024/01/08 18:18:37 ERROR : personal-stuff/longfilename2.jpg: Failed to copy: open /mnt/remotes/backup/files/2hc1kc2773q6o2gat2p4mji4lo/i8uj0rmsqs09kbahmasv2opduc/0vh6533kn5l82mpvlj93qdlegk/ufrlpp2l6pcdu2dpt08b6afm863ai7rnsapktibdb6t5adc0q683u2t0kka0ae7lpoqmbiqo2g7ed0mr4a2oej89m0ouk4nmbsgeahibl4ehs8461jf2quhs92ke29levje1jgc00g8eiecmea6o71f06nq62abnbcjejgalgmh4rgejb435hkafjg1uk2bj0cnkfhjv61l7vfl9r00momv9fnrfh51phkg7quomeesmfgfhrg5ih4g5vbhmki92: file name too long
2024/01/08 18:18:38 ERROR : personal-stuff/longfilename3.jpg: Failed to copy: open /mnt/remotes/backup/files/2hc1kc2773q6o2gat2p4mji4lo/i8uj0rmsqs09kbahmasv2opduc/1r6pj5g4ildaaam55dd2ekkuh0/r8mud097osv1l8pj05cucl3tk0/s4pnua0f70hng05ft28h2ecvouhq8hfle72962cu2si1b0sshubsbeetepgtec464vk8ipjafmud08fr3pf8qmvh7u313tpi1954lf0ms19a90a0vd3k3ajs7gbi6los0d0o79btfu9plaln1gvs7lcegno25thdj9p2oun2jmbna30vbectn0eeq950nrevl874avabm8c0kk3brtsu6nrdrt14lck7ebjshu4ttjod2bv7i03lbsgsanf7a08ll7a30hqblm3hriq1oecdrpeap2s3n51nkkeijudlo4shg6bp9mv6el7mgtbcqrcktl65n5agdubnce73j2q0ojnsvkn787et143ucv185haf5s1iddbr8d7p0sk84d3f: file name too long
2

You have names which are 386 characters long.

and this error message is expected for most filesystems.

3

the fact that the smb is remote, can be slowing things down.

4

I know. The issue is nothing to do with the latest log entries or the failed copies which I am fine with.

The issue is it hasn't done anything for hours and won't finish the sync (albeit with a few errors).

5

It did the first 500,000 checks in ~7 hours. It's done precisely zero checks since then.

The progress is still counting up the elapsed time with nothing more done since those old log entries.

image
image496×142 3.65 KB

6

that is a small amount of files to process.

i agree but so far, you only posted a snippet about file name too long
so hard to know what the real issue is.

lacking details but perhaps, if the smb is off-site/remote/vpn, could be part of the issue, slow, high latency, etc...
fwiw, rclone does support smb remotes, might test with that.
tho i do not recommend using it in production.

7

Exactly. It shouldn't take ten hours to check half a million files.

The smb is remote but has been rock solid and saturates a 1Gbps link if allowed.

I just don't get why a sync check is a) taking so long and b) freezing before the end.

The rest of the log also looks completely normal.

8

Also why can't crypt hash long filenames to prevent the filenames issue?

9

I ran it again overnight. This time it has frozen on check 507,333 after running for 12+ hours. Nothing interesting in the -vv log.

10

Can you kill -QUIT the rclone process when it appears to have got stuck.

This will produce a backtrace in the log - can you attach that here (or somewhere else - eg pastebin) so I can take a look at it. I don't need any of the log data, only the backtrace at the end.

With that I can tell you what rclone is doing when it gets stuck!

11

That doesn't seem to stop the process. It's still ticking away now at 18h3m23s.

12

omg! It just woke up and started doing something. 19h hours in.

Still no luck sending it kill commands an hour or so ago and had just left it since.

Unclear what it's doing because it's now at 680,000 checks and there aren't that many files according to ncdu.

Log looks normal, just chugging away.

The log from when it woke up:

2024/01/09 11:18:44 DEBUG : file.jpg: Unchanged skipping
2024/01/09 11:18:44 DEBUG : file2.jpg: Unchanged skipping
2024/01/09 17:35:09 DEBUG : PC/Windows/ServiceProfiles/NetworkService/AppData/Local/Microsoft/Media Player/Art Cache/LocalMLS/{00008D06-ABF8-41C1-B6BA-AA63BE7F3E51}.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
2024/01/09 17:35:09 DEBUG : PC/Windows/ServiceProfiles/NetworkService/AppData/Local/Microsoft/Media Player/Art Cache/LocalMLS/{00008D06-ABF8-41C1-B6BA-AA63BE7F3E51}.jpg: Unchanged skipping
2024/01/09 17:35:09 DEBUG : PC/Windows/ServiceProfiles/NetworkService/AppData/Local/Microsoft/Media Player/Art Cache/LocalMLS/{000044FD-395C-4D00-8551-766D1CA984DB}.jpg: Size and modification time the same (differ by 0s, within tolerance 1s)
13

It is not question of luck.. you are doing something wrong.

get PID of your rclone process and send QUIT signal as root. Then it will work for sure.

But given that it is moving again maybe it is not needed:) Can be that it is not rclone but for example your NAS? Running scrub or something similar making it unresponsive for some time?

14

I ran kill -QUIT 26725 which is the process shown by ps auxw

root 26725 0.0 0.0 6916 3216 pts/9 S+ Jan08 0:00 /bin/bash /usr/sbin/rclone sync /mnt/files crypt:/files --backup-dir=crypt:/.archive/files/2024/2024-01-08_11:36:22 --log-file=/mnt/rclone/rclone.log -P --modify-window=1s --filter-from=/mnt/rclone/filter_rules --transfers=8 -vv

Nothing happens. The process keeps running. rclone elapsed time keeps incrementing.

sudo kill -QUIT 26725 similarly does nothing.

What should I do differently?

I don't run any scrub and have never had issues running rclone for multiple days in a row previously.

Currently stalled at 21h and 1,125,396 checks.

15

why sync sync?

maybe the smb server needs a reboot?
maybe there is an optimization issue with filter_rules?

16

might be some quirk of your Linux distro... slackware likes to be different:)

sudo kill -3 26725

should work always.

17

Just a thought, if you are using Unraid, which it looks like, you can do something like Duplicacy for $20 for a true backup / block level program. I tend to only use rclone for cloud stuff.

18

Block level would be nice but would it be much quicker than rclone to a remote backup machine. I do prefer rclone in many ways.

19

typo, only one sync

Afraid that has also not done anything.

20

Have to do little testing but it being block level, I would imagine it would be. I started to use it to just backup my app data on my Unraid to my OneDrive, which I have a 1TB of data and man, it just works.

I'm annoyed I didn't change to Unraid much sooner for my home needs. My goal is what wastes the least amount of my time and just works all the time for a reasonable price and this checks all the boxes for me.

2 Likes