Copying files surges in write speed but then slows

Help and Support
1

What is the problem you are having with rclone?

Copying files with rclone becomes really slow at times but suddenly peaks in write speed for a few seconds.
The maximum write speed was about 35 megabytes per second, but it gradually decreases to a few kilobytes. Sometimes, it dips to 0 bytes per second but surges back to about 1 megabyte per second.

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.1-DEV
- os/version: debian trixie/sid (64 bit)
- os/kernel: 6.12.22-rt-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.2
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

None (From local to a flash drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --multi-thread-streams 32 --transfers 10 --copy-links --metadata --progress /home/horsey_guy/For_Everything /media/horsey_guy/Ventoy/For_Everything

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

Usage:
  rclone config [flags]
  rclone config [command]

Available Commands:
  create      Create a new remote with name, type and options.
  delete      Delete an existing remote.
  disconnect  Disconnects user from remote
  dump        Dump the config file as JSON.
  file        Show path of configuration file in use.
  password    Update password in an existing remote.
  paths       Show paths used for configuration, cache, temp etc.
  providers   List in JSON format all the providers and options.
  reconnect   Re-authenticates user with remote.
  show        Print (decrypted) config file, or the config for a single remote.
  touch       Ensure configuration file exists.
  update      Update options in an existing remote.
  userinfo    Prints info about logged in user of remote.

Flags:
  -h, --help   help for config

Additional help topics:
  rclone config edit       Enter an interactive configuration session.

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.
Command config needs 0 arguments maximum: you provided 1 non flag arguments: ["redacted"]

The command I ran took almost 5 hours to finish with a 7.9 gigabyte directory.

2

welcome to the forum,

that is a many years old, custom compiled dev version from an very out of date repository.

  1. can uninstall that version
  2. https://rclone.org/install/#script-installation
  3. test again using a debug log
3

Will the package be updated on Debian?

4

rclone has no control of that, need to ask the maintainers of that repository.
https://rclone.org/install/#package-manager

5

Alright, I reported a bug with reportbug to that package.

6

Same issue occurs with the latest version.

7

However, the speeds are a bit faster and does not dip to 0, but it still surges to 5 MiB/s and dips to 100 KiB/s. In the first few seconds, speeds went to 30 MiB/s completing the first gigabyte quickly.

8

remove these two flags. You can even try --transfers 2. High parallelism when writing often can have opposite results then expected.

9

Slowdowns still occur.

10

Given that it is fully local operation it is unlikely rclone issue. Most likely your flash drive is slow - it is old, lack TRIM or something similar and can sustain reasonable write speed for very short time.

To validate try to copy some larger set of files without rclone - I suspect it will behave exactly the same.

11

I probably suspect that my flash drive sustains fast write speeds for only a brief amount of time. Using rsync did have more stable write speeds though.

12

Because rsync is not using any parallelism and slower sometimes means faster overall:)

On the other hand the main reason people use rclone for local operations is exactly its ability to support parallelism. As long as your device has enough throughput capacity you can achieve much faster transfer speeds.

But at the end you have to chose what suits your configuration the best.

13

Why does rclone initially transfer one gigabyte of data within a minute but then takes hours to complete the remaining five gigabytes? Also, the large files over 200 megabytes do not seem to transfer until it is one of the last ones to be transferred.

14

If not happy with default order use --order-by string flag

15

Another method would be to manually copy individual large files.

16

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.