Rclone copy stuck on small files

Help and Support
1

Hi, I am trying to copy nfs shares between two storage accounts in Azure, we do it by mounting the shares to a container and then running rclone copy. The issue is rclone is getting stuck on some small files for hours, then it eventually succeeds, but it needs 9h for 9mb for example. Do you have any suggestions? Maybe a timeout or do we need to specifically skip those files?

Version:

rclone v1.67.0
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 5.15.0-1068-azure (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.4
- go/linking: static
- go/tags: none

Command:

rclone copy /mount/$NFS_SOURCE/ /mount/$NFS_DEST/ --progress --use-mmap --size-only -vvv --multi-thread-streams=$NUM_THREADS --transfers=64 --checkers=24 --buffer-size=256M

A log from the command i am trying to run with the -vv flag

Elapsed time:     21m43.5sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     21m44.0sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     21m44.5sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     21m45.0sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     21m45.5sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     21m46.0sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
2

welcome to the forum,

always getting stuck on the same files or is it random files?

what you posted was the snippet from the output of --progress which is not going to provide the answer.

please post the output of rclone config redacted

need to check the debug output, or might be easier to use a debug log
--log-level=DEBUG --log-file=rclone.log

--multi-thread-streams=$NUM_THREADS --transfers=64 --checkers=24 --buffer-size=256M
as a test, i would remove that. use defaults.

not fully understanding your setup, but if the source and dest are the same provider, might try --server-side-across-configs

3

Always getting stuck on the same files(same number).

root@rclone-manual-public-9vqr4:/scripts# rclone config redacted
2024/08/27 14:29:16 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
; empty config
### Double check the config for sensitive info before posting publicly

Providers are the same. Also we have tested with defaults, threads and transfers are not the problem.

4

Would be awesome to share the debug log.

5

ok, let's see if this is a rclone issue?

i would pick one trouble file.

  1. copy that single file, not using rclone. using cp, rsync or whatever copy tool.
  2. copy that single file, using rclone, post full debug log.
6

I am having problems recognising which files its getting stuck on. Everytime it shows a diffrent file on the last debug log.
Here is the debug log before it gets stuck

2024/09/03 11:59:26 DEBUG : downloads/invoices_controller/invoice-7580718.pdf: Unchanged skipping
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               777 / 778, 100%
Elapsed time:        11.4s
Checking:
 * downloads/invoices_con…er/invoice-7580718.pdf: checking
 * downloads/invoices_con…ller/invoice-98694.pdf: checking
2024/09/03 11:59:26 DEBUG : downloads/invoices_controller/invoice-98694.pdf: Unchanged skipping
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               778 / 778, 100%
Elapsed time:        11.4s
Checking:
 * downloads/invoices_con…ller/invoice-98694.pdf: checkingTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:        11.5sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -

This is the command i am using right now:

rclone copy /mount/$NFS_SOURCE/ /mount/$NFS_DEST/ --progress --use-mmap --log-level=DEBUG --size-only --multi-thread-streams=$NUM_THREADS --transfers=64 --checkers=24 --buffer-size=256M --server-side-across-configs

It seems it just gets stuck at the last two files?

7

i would test another copy tool, such as rsync

8

Here is the new Debug message we caught. Is that of help? I guess its not a file but a whole directory, which is why its slow at the end?

Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : Added delayed dir = "uploads/tmp/1725174820-212136339117336-2814-7171", newDst=uploads/tmp/1725174820-212136339117336-2814-7171
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : Added delayed dir = "uploads/tmp/1725289156-585085793117925-0005-9571", newDst=uploads/tmp/1725289156-585085793117925-0005-9571
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : Added delayed dir = "uploads/tmp/1725291108-39082547727673-0011-0145", newDst=uploads/tmp/1725291108-39082547727673-0011-0145
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : Added delayed dir = "uploads/tmp/1725300169-297570249273161-0006-1905", newDst=uploads/tmp/1725300169-297570249273161-0006-1905
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : Added delayed dir = "uploads/tmp/20121002-1550-8332-0598", newDst=uploads/tmp/20121002-1550-8332-0598
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 779, 100%
Elapsed time:     2h1m0.8s
2024/09/03 14:00:16 DEBUG : uploads/tmp/t.txt: Sizes identical
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 780, 100%
Elapsed time:     2h1m0.8s
Checking:
 *                             uploads/tmp/t.txt: checking
2024/09/03 14:00:16 DEBUG : uploads/tmp/t.txt: Unchanged skipping
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               779 / 780, 100%
Elapsed time:     2h1m0.8s
Checking:
 *                             uploads/tmp/t.txt: checkingTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               780 / 780, 100%
Elapsed time:     2h1m1.0sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               780 / 780, 100%
Elapsed time:     2h1m1.5sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               780 / 780, 100%
Elapsed time:     2h1m2.0sTransferred:                    0 B / 0 B, -, 0 B/s, ETA -
Checks:               780 / 780, 100%
9

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.