Rsync from crypt stalls while lookup is ongoing

What is the problem you are having with rclone?

Rsync seems to stall intermittently, after a couple of minutes it will continue. This is running on a mount which sonarr and radarr also access, while looking at the logs while the stall is ongoing it looks like it's doing lookups for one of those programs.

Running a test mount that only rsync uses seems to work better with no more stalling.
running another rsync on the same test mount which will do a lookup will make it stall as well.

What is your rclone version (output from rclone version)

1.53.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Team drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rsync -ahvP /home/x/.cloudcrypt/X /home/x/local/

The rclone config contents with secrets removed.

[clouddrive]
type = drive
scope = drive
service_account_file = SA1.json
team_drive = *
chunk_size = 64M

[cloudcrypt]
type = crypt
remote = clouddrive:Crypt2
filename_encryption = standard
password = *
password2 = *

[Unit]
Description=
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
User=X
Group=X
ExecStart=/usr/bin/rclone mount \
        --config /home/X/.config/rclone/rclone.conf \
        --read-only \
        --allow-other \
        --allow-non-empty \
        --dir-cache-time=96h \
        --timeout 1h \
        --rc \
        --rc-no-auth \
        --log-level=DEBUG \
        --log-file=/home/X/logs/clouddrive.log \
        cloudcrypt: /home/X/.cloudcrypt
ExecStop=/bin/fusermount -uz /home/X/.cloudcrypt
Restart=on-abort

[Install]
WantedBy=default.target

A log from the command with the -vv flag

2020/09/16 15:02:30 DEBUG : &{TVShowsCurrent/The Mentalist/Season 3/The Mentalist - S03E16 - Red Queen WEBDL-1080p.mkv (r)}: >Read: read=131072, err=<nil>
2020/09/16 15:02:30 DEBUG : &{TVShowsCurrent/The Mentalist/Season 3/The Mentalist - S03E16 - Red Queen WEBDL-1080p.mkv (r)}: >Read: read=131072, err=

Why are you using a mount and rsyncing rather than using rclone sync to local to the remote? That would be much more effecient.

To be honest? Not sure actually :wink:

To be fair my mounts are all merged with mergerfs with local storage so 99% of the time i'm using rsync for all moves, it's only now that I need to move a large amount of data local (and I've got debug logs going) that I started to notice the stalls.
Running rclone copy now with 8 transfers and I can max out my 10Gb link so that's good. While I'm curious for the cause of the stalls I don't think it's important as rclone copy would suffice.

The logs would show what's going on if there is something in rclone going on.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.