Transfer 10x the size of the files synced

Help and Support
1

I’m not sure if I’ve configured something incorrectly but when running a sync command the size of the transfer seems excessively large.

I ran the following command:

rclone sync -v --transfers=32 --log-file=/var/log/backblaze-home.log /home/ backblaze:Fasthosts-HOST2/home/

There were about 6000 files that were either new or changed in a directory containing 109MB of files.

Here’s the final summary of the sync:

Transferred: 1.127 GBytes (333.209 kBytes/s)
Errors: 0
Checks: 839427
Transferred: 6201
Elapsed time: 59m7.9s

Given that there could have only been a maximum of 109MB of files to transfer, why did the actual transfer total 1.127GB?

I can supply the full log if needed.

Jeremy

2

That does seem strange. Can you send the log? You can send it to nick@craig-wood.com if you don’t want to post it publically - please put a link to this issue in the message otherwise it will get lost in the maelstrom of chaos that is my INBOX!

3

Hey - thank you for the quick response :slight_smile:

I’ll email you the log now.

Thank you again.

4

I’ve had a look at that.

No files were uploaded many times which is good.

What it look like is that it decided to re-upload all your files.

Have the timestamps changed on them? If the timestamps change regularly then you’ll want to use the --checksum flag.

What OS are you running? Could this be related to Daylight Savings time change I wonder?

5

I’m running Centos 6.7.

Don’t think daylight savings would be in effect here? I only installed Rclone 3 days ago.

I’ll try the checksum flag :slight_smile:

6

Unlikely indeed.

How did you upload the original files - with rclone? Or a different tool?

7

I used Rclone sync to upload the original files

8

I think the modtimes must have changed then between runs - that would be my guess. Does some process write those files regularly?

9

No. The 6200 odd files were updated during an update of Wordpress on one website.

10

If you run with -vv rclone will tell you exactly why it uploads something

11

Ah OK - I’ll do that

Thank you!