Migrated from teamdrive to purchased google drive space

droidly · July 21, 2022, 2:36am

What is the problem you are having with rclone?

I use rclone with Plex on windows 10 for almost a year now without any problems. Since moving my data to the paid space my Plex scans are so slow they are almost useless. As a comparison prior to the move my movies all in one folder no sub-folders would scan in 10 seconds. Now it is hours and it may take months at this pace.

Run the command 'rclone version' and share the full output of the command.

Which cloud storage system are you using? (eg Google Drive)

google drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount --rc --cache-dir=C:\Users\andre\AppData\Local\rclone --bwlimit-file 5M --vfs-read-chunk-size=1M --vfs-read-chunk-size-limit=0 --tpslimit 10 --tpslimit-burst 10 --vfs-cache-max-age 72h --dir-cache-time 72h --drive-acknowledge-abuse --vfs-cache-mode full gdrive:/ d:/gdrive

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = "removed"
client_secret = "removed"
scope = drive.readonly
token = "removed"
root_folder_id = "removed"
team_drive =

A log from the command with the `-vv` flag

D:\Software\Rclone\rclone>rclone mount -vv --rc --cache-dir=C:\Users\andre\AppData\Local\rclone --bwlimit-file 5M --vfs-read-chunk-size=1M --vfs-read-chunk-size-limit=0 --tpslimit 10 --tpslimit-burst 10 --vfs-cache-max-age 72h --dir-cache-time 72h --drive-acknowledge-abuse --vfs-cache-mode full gdrive:/ d:/gdrive
2022/07/20 19:28:16 INFO  : Starting transaction limiter: max 10 transactions/s with burst 10
2022/07/20 19:28:16 DEBUG : rclone: Version "v1.59.0" starting with parameters ["rclone" "mount" "-vv" "--rc" "--cache-dir=C:\\Users\\andre\\AppData\\Local\\rclone" "--bwlimit-file" "5M" "--vfs-read-chunk-size=1M" "--vfs-read-chunk-size-limit=0" "--tpslimit" "10" "--tpslimit-burst" "10" "--vfs-cache-max-age" "72h" "--dir-cache-time" "72h" "--drive-acknowledge-abuse" "--vfs-cache-mode" "full" "gdrive:/" "d:/gdrive"]
2022/07/20 19:28:16 NOTICE: Serving remote control on http://localhost:5572/
2022/07/20 19:28:16 DEBUG : Creating backend with remote "gdrive:/"
2022/07/20 19:28:16 DEBUG : Using config file from "D:\\Software\\Rclone\\rclone\\rclone.conf"
2022/07/20 19:28:16 DEBUG : gdrive: detected overridden config - adding "{P2dbo}" suffix to name
2022/07/20 19:28:16 DEBUG : fs cache: renaming cache item "gdrive:/" to be canonical "gdrive{P2dbo}:"
2022/07/20 19:28:16 DEBUG : vfs cache: root is "C:\\Users\\andre\\AppData\\Local\\rclone"
2022/07/20 19:28:16 DEBUG : vfs cache: data root is "\\\\?\\C:\\Users\\andre\\AppData\\Local\\rclone\\vfs\\gdrive{P2dbo}"
2022/07/20 19:28:16 DEBUG : vfs cache: metadata root is "\\\\?\\C:\\Users\\andre\\AppData\\Local\\rclone\\vfsMeta\\gdrive{P2dbo}"
2022/07/20 19:28:16 DEBUG : Creating backend with remote "C:/Users/andre/AppData/Local/rclone/vfs/gdrive{P2dbo}/"
2022/07/20 19:28:16 DEBUG : fs cache: renaming cache item "C:/Users/andre/AppData/Local/rclone/vfs/gdrive{P2dbo}/" to be canonical "//?/C:/Users/andre/AppData/Local/rclone/vfs/gdrive{P2dbo}/"
2022/07/20 19:28:16 DEBUG : Creating backend with remote "C:/Users/andre/AppData/Local/rclone/vfsMeta/gdrive{P2dbo}/"
2022/07/20 19:28:16 DEBUG : fs cache: renaming cache item "C:/Users/andre/AppData/Local/rclone/vfsMeta/gdrive{P2dbo}/" to be canonical "//?/C:/Users/andre/AppData/Local/rclone/vfsMeta/gdrive{P2dbo}/"
2022/07/20 19:28:16 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2022/07/20 19:28:16 DEBUG : Network mode mounting is disabled
2022/07/20 19:28:16 DEBUG : Mounting on "d:\\gdrive" ("gdrive{P2dbo}")
2022/07/20 19:28:16 DEBUG : Google drive root '': Mounting with options: ["-o" "attr_timeout=1" "-o" "uid=-1" "-o" "gid=-1" "--FileSystemName=rclone" "-o" "volname=gdrive{P2dbo}"]
2022/07/20 19:28:16 DEBUG : Google drive root '': Init:
2022/07/20 19:28:16 DEBUG : Google drive root '': >Init:
2022/07/20 19:28:16 DEBUG : /: Statfs:
2022/07/20 19:28:16 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:1342177280 Bfree:181624426 Bavail:181624426 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2022/07/20 19:28:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/07/20 19:28:16 DEBUG : /: >Getattr: errc=0
2022/07/20 19:28:16 DEBUG : /: Readlink:
2022/07/20 19:28:16 DEBUG : /: >Readlink: linkPath="", errc=-40
2022/07/20 19:28:16 DEBUG : /: Getxattr: name="non-existant-a11ec902d22f4ec49003af15282d3b00"
2022/07/20 19:28:16 DEBUG : /: >Getxattr: errc=-40, value=""
The service rclone has been started.
2022/07/20 19:28:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/07/20 19:28:16 DEBUG : /: >Getattr: errc=0
2022/07/20 19:28:16 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2022/07/20 19:28:16 DEBUG : /: >Getattr: errc=0
2022/07/20 19:28:16 DEBUG : /: Opendir:
2022/07/20 19:28:16 DEBUG : /: OpenFile: flags=O_RDONLY, perm=-rwxrwxrwx
2022/07/20 19:28:16 DEBUG : /: >OpenFile: fd=/ (r), err=<nil>
2022/07/20 19:28:16 DEBUG : /: >Opendir: errc=0, fh=0x0
2022/07/20 19:28:16 DEBUG : /: Statfs:
2022/07/20 19:28:16 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:1342177280 Bfree:181624426 Bavail:181624426 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2022/07/20 19:28:16 DEBUG : /: Statfs:
2022/07/20 19:28:16 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:1342177280 Bfree:181624426 Bavail:181624426 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
2022/07/20 19:28:16 DEBUG : /: Releasedir: fh=0x0
2022/07/20 19:28:16 DEBUG : /: >Releasedir: errc=0
2022/07/20 19:29:16 DEBUG : Google drive root '': Checking for changes on remote
2022/07/20 19:29:16 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2022/07/20 19:30:16 DEBUG : Google drive root '': Checking for changes on remote
2022/07/20 19:30:16 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2022/07/20 19:30:16 DEBUG : gdrive{P2dbo}: Loaded invalid token from config file - ignoring
2022/07/20 19:30:16 DEBUG : Saving config "token" in section "gdrive" of the config file
2022/07/20 19:30:16 DEBUG : Keeping previous permissions for config file: -rw-rw-rw-
2022/07/20 19:30:16 DEBUG : gdrive{P2dbo}: Saved new token in config file
2022/07/20 19:31:16 DEBUG : Google drive root '': Checking for changes on remote
2022/07/20 19:31:16 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)

VBB · July 21, 2022, 10:49am

Have a look here Slow scan plex tv show, and let us know how it goes.

asdffdsa · July 21, 2022, 12:26pm

hi
might want to remove --bwlimit-file 5M

droidly · July 21, 2022, 1:47pm

hi, thanks for the reply. I will remove it and test it. I only added that --bwlimit-file 5M because I saw someone else suggested it. My overall knowledge of rclone is somewhat limited but it has been working great for me now with PLEX for almost a year.

Thanks

droidly · July 21, 2022, 1:54pm

Hi VBB, thanks for the link. I let PLEX run all night and both movies and tv sync'd and updated all metadata in plex. It is working but my scans are slower for some reason. I have all of those recommendations already. I am going to try asdffdsa's suggestion and see if that helps.

Thanks again, Andre

asdffdsa · July 21, 2022, 1:57pm

i would remove --tpslimit 10 --tpslimit-burst 10

and mimic this
https://github.com/animosity22/homescripts/blob/master/systemd/rclone-drive.service

VBB · July 21, 2022, 2:04pm

I think rclone rc vfs/refresh recursive=true is key. Did you implement that step?

Also, are you scanning for the first time (new library)? If so, that is always a lot slower than subsequent scans.

droidly · July 21, 2022, 2:08pm

yes I always do rclone rc vfs/refresh recursive=true just after I mount the drive. It's not a new library 4 TB but I did MOVE not copy all of the files from the teamdrive to google one drive 5 TB limit.

I will implement asdffdsa's suggestions and let y'all know shortly. Thanks

droidly · July 21, 2022, 2:53pm

Hi again,

I mimiced all of animosity22's settings except for a few things that do not apply to me. My movies and tv episodes start much faster, in fact almost instantly whereas before it was about 10s. It looks like this

rclone mount --rc --dir-cache-time 5000h --poll-interval 10s --cache-dir=C:\Users\andre\AppData\Local\rclone --drive-pacer-min-sleep 10ms --drive-pacer-burst 200 --vfs-cache-mode full --vfs-cache-max-size 250G --vfs-cache-max-age 5000h --vfs-cache-poll-interval 5m --bwlimit-file 32M gdrive:/ d:/gdrive

This is my my rclone cache settings, do you recommend any changes?

rclone rc vfs/refresh recursive=true --drive-pacer-burst 200 --drive-pacer-min-sleep 10ms --timeout 30m

Thanks again, Andre

asdffdsa · July 21, 2022, 2:56pm

looks good to me

droidly · July 21, 2022, 3:13pm

thanks again, I will consider this resolved.

system · July 24, 2022, 3:14pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.