Slow loading speed on mounting (Windows)

Help and Support
1

What is the problem you are having with rclone?

Slow loading speed when mounting Drive on Windows (11). When I access to a folder which has lots of subfolders or file (even just access to a shared drive), it is very slow. Sometimes my explorer crashes bcs of it. Could you guys help me to optimise to load files faster?

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

  • os/version: Microsoft Windows 11 Home Single Language 22H2 (64 bit)
  • os/kernel: 10.0.22621.1413 Build 22621.1413.1413 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.20.2
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

GDrive and OneDrive (Mega + Dropbox also)
I use Combine Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount CombineAll: A: --cache-dir "D:\Program Files\Rclone Cache" --vfs-cache-mode writes --cache-info-age --vfs-cache-max-age 48h --network-mode --volname "Cloud Storages" --no-modtime --no-checksum --buffer-size=18 -v

The rclone config contents with secrets removed.

[Dropbox]
type = dropbox
token = *****

[Mega]
type = mega
user = *****
pass = *****

[onedrive NHH]
type = onedrive
token = *****
drive_id = *****
drive_type = business

[GDrive46]
type = drive
scope = drive
token = *****
team_drive = 

[GGPhotos46]
type = google photos
read_only = true
token = *****

[OneDriveE5]
type = onedrive
token = *****
drive_id = *****
drive_type = business

[OneDrive46]
type = onedrive
token = *****
drive_id = *****
drive_type = personal

[GDriveMirror]
type = drive
scope = drive
token = *****
team_drive = *****
root_folder_id = 

[GDrive56]
type = drive
scope = drive
token = *****
team_drive = 

[GDrive7565]
type = drive
scope = drive
token = *****
team_drive = 

[GDriveConNgoanMirror]
type = drive
scope = drive
token = *****
team_drive = 

[OneDrive7565]
type = onedrive
token = *****
drive_id = *****
drive_type = personal

[GDriveDuck]
type = drive
scope = drive
token = *****
team_drive = 

[GDriveProton]
type = drive
scope = drive
token = *****
team_drive = 

[CombineAll]
type = combine
upstreams = "GDrive46=GDrive46:" "GDrive56=GDrive56:" "GDrive7565=GDrive7565:" "GDriveMirror=GDriveMirror:" "GDriveConNgoanMirror=GDriveConNgoanMirror:" "GDriveDuck=GDriveDuck:" "GDriveProton=GDriveProton:" "OneDrive46=OneDrive46:" "OneDrive7565=OneDrive7565:" "OneDriveE5=OneDriveE5:" "Mega=Mega:" "Dropbox=Dropbox:"

[CombineNeccessary]
type = combine
upstreams = "GDrive46=GDrive46:" "GDriveMirror=GDriveMirror:" "OneDrive46=OneDrive46:" "OneDriveE5=OneDriveE5:"

A log from the command with the -vv flag

Not relate to the question I ask :>
2

Best bet would be to not use explorer or turn off all the features.

It's slow because you have a bunch of remotes in the mix that are slow as you'd see those errors in the debug log.

4

I have tried to mount 1 GDrive only and its loading speed was slow same as mounting Combine Drive (only enable --vfs-cache-mode writes --network-mode). And it didn't show any errors in the log.

Edited: I used -vv and here is the log. It said Rate Limit Exceeded :((. It still loaded but slow.

2023/03/21 19:33:18 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'drive.googleapis.com' for consumer 'project_number:202264815644'.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/202264815644",
      "quota_limit": "defaultPerMinutePerProject",
      "quota_limit_value": "420000",
      "quota_location": "global",
      "quota_metric": "drive.googleapis.com/default",
      "service": "drive.googleapis.com"
    },
    "reason": "RATE_LIMIT_EXCEEDED"
  },
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Request a higher quota limit.",
        "url": "https://cloud.google.com/docs/quota#requesting_higher_quota"
      }
    ]
  }
]
, rateLimitExceeded)
2023/03/21 19:33:18 DEBUG : pacer: Rate limited, increasing sleep to 1.173388939s
2023/03/21 19:33:19 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'drive.googleapis.com' for consumer 'project_number:202264815644'.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/202264815644",
      "quota_limit": "defaultPerMinutePerProject",
      "quota_limit_value": "420000",
      "quota_location": "global",
      "quota_metric": "drive.googleapis.com/default",
      "service": "drive.googleapis.com"
    },
    "reason": "RATE_LIMIT_EXCEEDED"
  },
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Request a higher quota limit.",
        "url": "https://cloud.google.com/docs/quota#requesting_higher_quota"
      }
    ]
  }
]
, rateLimitExceeded)
2023/03/21 19:33:19 DEBUG : pacer: Rate limited, increasing sleep to 2.546596426s
2023/03/21 19:33:20 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'drive.googleapis.com' for consumer 'project_number:202264815644'.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/202264815644",
      "quota_limit": "defaultPerMinutePerProject",
      "quota_limit_value": "420000",
      "quota_location": "global",
      "quota_metric": "drive.googleapis.com/default",
      "service": "drive.googleapis.com"
    },
    "reason": "RATE_LIMIT_EXCEEDED"
  },
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Request a higher quota limit.",
        "url": "https://cloud.google.com/docs/quota#requesting_higher_quota"
      }
    ]
  }
]
, rateLimitExceeded)
2023/03/21 19:33:20 DEBUG : pacer: Rate limited, increasing sleep to 4.946049554s
2023/03/21 19:33:23 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'drive.googleapis.com' for consumer 'project_number:202264815644'.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/202264815644",
      "quota_limit": "defaultPerMinutePerProject",
      "quota_limit_value": "420000",
      "quota_location": "global",
      "quota_metric": "drive.googleapis.com/default",
      "service": "drive.googleapis.com"
    },
    "reason": "RATE_LIMIT_EXCEEDED"
  },
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Request a higher quota limit.",
        "url": "https://cloud.google.com/docs/quota#requesting_higher_quota"
      }
    ]
  }
]
, rateLimitExceeded)
2023/03/21 19:33:23 DEBUG : pacer: Rate limited, increasing sleep to 8.633158653s
2023/03/21 19:33:27 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: Quota exceeded for quota metric 'Queries' and limit 'Queries per minute' of service 'drive.googleapis.com' for consumer 'project_number:202264815644'.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.ErrorInfo",
    "domain": "googleapis.com",
    "metadata": {
      "consumer": "projects/202264815644",
      "quota_limit": "defaultPerMinutePerProject",
      "quota_limit_value": "420000",
      "quota_location": "global",
      "quota_metric": "drive.googleapis.com/default",
      "service": "drive.googleapis.com"
    },
    "reason": "RATE_LIMIT_EXCEEDED"
  },
  {
    "@type": "type.googleapis.com/google.rpc.Help",
    "links": [
      {
        "description": "Request a higher quota limit.",
        "url": "https://cloud.google.com/docs/quota#requesting_higher_quota"
      }
    ]
  }
]
, rateLimitExceeded)
2023/03/21 19:33:27 DEBUG : pacer: Rate limited, increasing sleep to 16.154107327s
2023/03/21 19:33:37 DEBUG : pacer: Reducing sleep to 0s
5

hello and welcome to the forum,

none of your remotes have a client id/secret, forcing rclone to use its default values, resulting in poor performance.
in addition, other non-rclone projects are also using the default client id.

When you use rclone with Google drive in its default configuration you are using rclone's client_id. This is shared between all the rclone users. There is a global rate limit on the number of queries per second that each client_id can do set by Google

might try --vfs-cache-mode=full, as that will cache reads, in addition to writes.

based on forum discussions, the recommend value is
--buffer-size=0

--cache-info-age - this flags does nothing. and even if it did something, does not have a value.

6

Thank you a lot. I will follow your help and respond the result later <3

7

will my Token be renew everytime I use rclone when using custom client id/secret?

8

yes. rclone will renew tokens on its own, as needed

1 Like
9

could you explain why we should keep --buffer-size=0? Thank you :smiling_face_with_three_hearts:

10

well, how did you choose --buffer-size=18 ?
for now, really does not matter one way or another.

first, i would create client id for gdrive/dropbox. not clear is that makes a difference with onedrive.
and mega is going to be very slow, no matter what you tweak. for testing, i would remove that.

1 Like
11

Ouhm just randomly.... I have seen somewhere that said set the --buffer-size=0. Then I felt it faster to load a video. I tried to set 18 and it didn't change much. After all I keep 18 for no reason :no_mouth:
By the way thank you. It loads file faster than default client (not as fast as I expected but I think it's fine).

12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.