Rclone mount to Google Drive fails

What is the problem you are having with rclone?

Rclone mount fails to behave as expected for a flat directory on Google Drive (100K ~50MB files), MacOS. Fails to populate directory cache. nfs server localhost:/repo: not responding Waited 1h+.

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.0-beta.7051.7a5491ba7

  • os/version: darwin 13.4 (64 bit)
  • os/kernel: 22.5.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.20.4
  • go/linking: dynamic
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

Google Drive.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount repo: ~/repo --vfs-cache-mode writes --volname repo --debug-fuse  --noapplexattr --noappledouble --allow-non-empty

The rclone config contents with secrets removed.

[qw]
type = drive
client_id = <redacted>
client_secret = <redacted>
scope = drive
token = <redacted>
team_drive = 
server_side_across_configs = true
root_folder_id = <redacted>

[repo]
type = crypt
remote = qw:repo/
password = <redacted>
password2 = <redacted>

A log from the command with the -vv flag

2023/05/24 11:23:31 DEBUG : Setting --exclude-from "[/Users/TimeMachineExclusionList.txt]" from environment variable RCLONE_EXCLUDE_FROM="/Users/TimeMachineExclusionList.txt"
2023/05/24 11:23:31 DEBUG : rclone: Version "v1.63.0-beta.7051.7a5491ba7" starting with parameters ["rclone" "mount" "repo:" "/Users/repo" "--vfs-cache-mode" "writes" "--volname" "Photos" "--debug-fuse" "--noapplexattr" "--noappledouble" "--allow-non-empty" "-vv"]
2023/05/24 11:23:31 DEBUG : Creating backend with remote "repo:"
2023/05/24 11:23:31 DEBUG : Using config file from "/Users/x/.rclone.conf"
2023/05/24 11:23:31 DEBUG : Creating backend with remote "qw:"
2023/05/24 11:23:32 DEBUG : fs cache: renaming cache item "qw:" to be canonical "qw:"
2023/05/24 11:23:32 DEBUG : vfs cache: root is "/Users/x/Library/Caches/rclone"
2023/05/24 11:23:32 DEBUG : vfs cache: data root is "/Users/x/Library/Caches/rclone/vfs/repo"
2023/05/24 11:23:32 DEBUG : vfs cache: metadata root is "/Users/x/Library/Caches/rclone/vfsMeta/repo"
2023/05/24 11:23:32 DEBUG : Creating backend with remote "/Users/x/Library/Caches/rclone/vfs/repo/"
2023/05/24 11:23:32 DEBUG : fs cache: renaming cache item "/Users/x/Library/Caches/rclone/vfs/repo/" to be canonical "/Users/x/Library/Caches/rclone/vfs/repo"
2023/05/24 11:23:32 DEBUG : Creating backend with remote "/Users/x/Library/Caches/rclone/vfsMeta/repo/"
2023/05/24 11:23:32 DEBUG : fs cache: renaming cache item "/Users/x/Library/Caches/rclone/vfsMeta/repo/" to be canonical "/Users/x/Library/Caches/rclone/vfsMeta/repo"
2023/05/24 11:23:32 DEBUG : Mounting on "/Users/repo" ("repo")
2023/05/24 11:23:32 DEBUG : Adding "-o modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC" for macOS
2023/05/24 11:23:32 DEBUG : Encrypted drive 'repo:': Mounting with options: ["-o" "attr_timeout=1" "-o" "debug" "-o" "fsname=repo:" "-o" "subtype=rclone" "-o" "max_readahead=131072" "-o" "atomic_o_trunc" "-o" "daemon_timeout=600" "-o" "volname=Photos" "-o" "noappledouble" "-o" "noapplexattr" "-o" "modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC"]
2023/05/24 11:23:32 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
FUSE library version: 2.9.9
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0
INIT: 7.18
flags=0xfe00007b
max_readahead=0x00000000
2023/05/24 11:23:32 DEBUG : Encrypted drive 'repo:': Init: 
2023/05/24 11:23:32 DEBUG : Encrypted drive 'repo:': >Init: 
   INIT: 7.19
   flags=0x00000019
   max_readahead=0x00000000
   max_write=0x02000000
   max_background=0
   congestion_threshold=0
   unique: 2, success, outsize: 40
unique: 3, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0
statfs /
2023/05/24 11:23:32 DEBUG : /: Statfs: 
2023/05/24 11:23:33 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
   unique: 3, success, outsize: 96
unique: 4, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 0
getattr /
2023/05/24 11:23:33 DEBUG : /: Getattr: fh=0xFFFFFFFFFFFFFFFF
2023/05/24 11:23:33 DEBUG : /: >Getattr: errc=0
   unique: 4, success, outsize: 136
unique: 5, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0
statfs /
2023/05/24 11:23:33 DEBUG : /: Statfs: 
2023/05/24 11:23:33 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
   unique: 5, success, outsize: 96
unique: 6, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0
statfs /
2023/05/24 11:23:33 DEBUG : /: Statfs: 
2023/05/24 11:23:33 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
   unique: 6, success, outsize: 96
unique: 7, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 0
statfs /
2023/05/24 11:23:33 DEBUG : /: Statfs: 
2023/05/24 11:23:33 DEBUG : /: >Statfs: stat={Bsize:4096 Frsize:4096 Blocks:4294967295 Bfree:4294967295 Bavail:4294967295 Files:1000000000 Ffree:1000000000 Favail:0 Fsid:0 Flag:0 Namemax:255}, errc=0
   unique: 7, success, outsize: 96

^C2023/05/24 11:23:53 INFO  : Signal received: interrupt
2023/05/24 11:23:53 DEBUG : Not calling host.Unmount as signal received
2023/05/24 11:23:53 DEBUG : vfs cache: cleaner exiting
2023/05/24 11:23:53 DEBUG : Unmounted successfully
2023/05/24 11:23:53 ERROR : /Users/repo: Unmounted rclone mount
2023/05/24 11:23:53 INFO  : Exiting...

Not 100% sure how fuse is on the Mac, but on Linux, you need fuse3.

You probably want to update that and try.

What does this output? Do you see your files? How long does it take?

rclone lsf repo:

How about this?

rclone lsf qw:

Are you using FUSE-T? If so can you try macFUSE?

What may be happening is that the directory listing is taking too long and the kernel is timing out.

Both commands list all the files just fine and take ~10 minutes to complete (maybe it would be way faster if it was not one of 3 parallel tasks associated with this API user/Drive account).

I will investigate MacFuse again. (earlier I tried both regular rclone macfuse and Fuse-T, but someone mentioned there is old version of macfuse in my logs, so I will investigate that further)

Timeout is plausible, sure, but I don't get it why the whole system seems to take a nose dive.

I hope you will migrate to MacOS hardware/environment :slight_smile:

That is probably the problem. The kernel will be waiting for rclone for 10 minutes and it will timeout.

You can try setting this flag

  --daemon-timeout Duration                Time limit for rclone to respond to kernel (not supported on Windows) (default 0s)

Say --daemon-timeout 15m which might help. (The default is 10 minutes on macOS). There is an upper limit for it which I can't remember but it isn't very big. You may also be able to turn off this timeout using --daemon-timeout 0 which might also be worth a try.

When stuff goes wrong in kernel space, the system tends to suffer.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.