Large downloads while uploading : Google Drive

Help and Support
1

What is the problem you are having with rclone?

Large downloads while uploading to Google Drive (crypt)

What is your rclone version (output from rclone version)

os/arch: linux/amd64
go version: go1.14.4

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Synology NAS

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy -Pv /volume1/Media/Movies/ /volume1/Media/Google/

The rclone config contents with secrets removed.

[google]
type = drive
client_id = (REMOVED)
client_secret = (REMOVED)
scope = drive
token = (REMOVED)

[GoogleCache]
type = cache
remote = google:
chunk_size = 50M
info_age = 1h0m0s
chunk_total_size = 20G

[GoogleCrypt]
type = crypt
remote = GoogleCache:
filename_encryption = standard
directory_name_encryption = true
password = (REMOVED)
password2 = (REMOVED)

A log from the command with the -vv flag

![rclone|664x500](upload://e4OszxmnGB9KN0wghXlFPr6pJxQ.png)

Bandwhich

The above shows bandwhich which rclone consuming approx 67MBps on downloads which is not expected?

Graph

The graph shown above also shows this activity, strangely it starts around 03:00 and then stops around 05:30.

Thanks

2

Please add the debug logs too (as was mentioned in the question template).

3

I’m going to have to kick the copy process off again as I stopped it, it didn’t have -vv flag set when I started it off.

In the absence of logs I just don’t understand why rclone should be consuming so much download bandwidth while doing an upload to Google Drive.

4

Are you copying to a mounted partition? I don't see a remote origin or destination in your copy command.

5

Yes, copying to a mounted partition

6

And you are sure nothing is reading from the mount?

Please add the debug logs for the mount too.

7

Yup pretty sure nothing should be reading from the mount, from the picture (Bandwhich hyperlink) you can see it's rclone not another process consuming 67Mbps of download bandwith, I'm expecting only upload of approx 4.5 Mbps

8

That doesn't mean nothing is reading from the mount unless the mount is not via rclone?

If it is with rclone, there is nothing in either of the graphs which indicate whether it's the rclone process for copy or mount.

9

Nothing is using the rclone mount other than the upload, as you can see from the bandwhich screen grab there are a number of rclone processes all pulling 15+Mbps

10

Yes, and that means something is reading from the mount which is why rclone is downloading it.

It can be the rclone copy process (doubtful) or it can be something else. We will need the debug logs for both the copy and the mount to get any more info. RClone doesn't download anything without being asked for it.

11

What is your mount command and path? I don't think rclone can do server copy if the copy is done on a mounted partition (I might be wrong) also based on your rclone mount command, it could be rclone is caching some information when copy command is trying to read the files and explain the download.

Also, if you are copying from local to remote mounted using rclone, you are far better off copying directly to the remote and avoid the mounted partition. This will avoid a lot of overhead, avoid parallel uploads and just be far more efficient.

So we need:

  1. mount command
  2. copy command
  3. debug log from mount
  4. debug log from copy

Otherwise any guess is good.

12

  • it should not be hard to figure out which process starts running at 03:00.
    and to figure out which process is using the bandwidth.
    did you check the Task Scheduler?

  • as i understand, rclone mount can do a server copy.

  • i would remove the cache backend, as it is almost never needed, buggy, and soon to be depreciated.
    https://rclone.org/cache/#status

13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.