High CPU & memory utilization of Windows 10 cmd promt

Help and Support
1

#### What is the problem you are having with rclone?
When copying files across to a Microsoft OneDrive remote or validating checksums, there is a significant increase in CPU and memory utilization.

The issue appears to occur when the command prompt has been active for 30 minutes or longer. It results in the inability to interact or navigate other active windows or applications e.g. Google Chrome. Alternatively, all other windows and/or applications take much longer to respond.

Once the cmd prompt process is stopped, the device returns to normal operations.

#### What is your rclone version (output from rclone version)
rclone v1.53.1

#### Which OS you are using and how many bits (eg Windows 7, 64 bit)
Microsoft Windows 10 Professional version 1909 Build 18636.1110

#### Which cloud storage system are you using? (eg Google Drive)
Microsoft OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone.exe --stats 10s --progress --bwlimit 7M --checksum --log-file=rclone_log.txt --log-level DEBUG -copy /path/to/files/on/local/hard/drive remote:/ --backup-dir=remote:/path

rclone.exe --stats 10s --progress --bwlimit 7M --log-file=rclone_log.txt --log-level DEBUG -copy /path/to/files/on/local/hard/drive remote:/ --backup-dir=remote:/path

rclone.exe check --stats 10s --progress --log-file=rclone_log.txt --log-level DEBUG /path/to/files/on/local/hard/drive remote:/path

The rclone config contents with secrets removed.

The configuration is standard i.e. it follows the Microsoft OneDrive rclone guide - https://rclone.org/onedrive/. There is no encryption configured.

A log from the command with the -vv flag

There is no additional information in the log files in additional to operational events e.g. successful file commit.

2

Can you share the debug log please?

3

to better understand what is the cause, you can test with these flags.

this flag will perform the checks before transfer, to see if the hard drive is the limiting factor
https://rclone.org/docs/#check-first
"This flag can be useful on IO limited systems where transfers interfere with checking."

try reducing https://rclone.org/docs/#checkers-n from the default 8 to 2.

4

@Animosity022, what part of the debug log would be helpful?

The only events are multipart start, segment uploads, acknowledgments (OK), source and destination SHA-1 checks, status (Unchanged skipping).

5

Thanks @asdffdsa. If the flag --checkers is reduced from 8 to 2, what behavior is expected? For example, will there be a visible difference to CPU & memory utilization?

6

give it a try, every system is different.

as first, i did a lot of testing, tweaking --checkers and --transfers to find the optimum values.
take a set of files that represent what you normally transfer/check
then run rclone, tweaks the settings and see how long it takes, how much ram and cpu.

my backup server uses the awesome REFS file system, windows version of zfs.
it has three slow mechanical hard drives in a software raid5.
i use rclone to copy large veeam backup files to cloud.

7

The log would be helpful as that's how I'd debug / assist. Without seeing the log, it's hard to guess or assume what the issue would be.

8

Thanks @asdffdsa. I'll report back. It would be great anyone has tested across different systems and published the results.

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.