Rclone Mount Issues

CableGuySteve · March 31, 2023, 12:37pm

What is the problem you are having with rclone?

When I run this command - rclone mount --daemon --allow-other Photos:website_photos /home/ch/public_html/photos_rclone - to serve images to my website from Google Drive - most of the images will appear with no issue but some photos will not load at all and the page just says loading...

Example:

https://canadianheadstones.ca/photos_rclone/on/photos/county029/c029_victoria_lawn_c_id128221.jpg - Will not load
https://canadianheadstones.ca/photos_rclone/on/photos/county017/c017_glen_williams_c_id585.jpg - Loads right away

There are some photos that will load but takes about 10 to 20 secs to load.

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2

os/version: centos 7.9.2009 (64 bit)
os/kernel: 3.10.0-1160.83.1.el7.x86_64 (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.20.2
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone mount --daemon --allow-other Photos:website_photos /home/ch/public_html/photos_rclone

The rclone config contents with secrets removed.

Paste config here

A log from the command with the `-vv` flag

2023/03/31 08:32:14 DEBUG : Google drive root 'website_photos': Mounting on "/home/ch/public_html/photos_rclone"
2023/03/31 08:32:14 DEBUG : : Root:
2023/03/31 08:32:14 DEBUG : : >Root: node=/, err=<nil>
2023/03/31 08:32:22 DEBUG : /: Lookup: name="on"
2023/03/31 08:32:23 DEBUG : /: >Lookup: node=on/, err=<nil>
2023/03/31 08:32:23 DEBUG : on/: Attr:
2023/03/31 08:32:23 DEBUG : on/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:32:23 DEBUG : on/: Lookup: name="photos"
2023/03/31 08:32:23 DEBUG : on/: >Lookup: node=on/photos/, err=<nil>
2023/03/31 08:32:23 DEBUG : on/photos/: Attr:
2023/03/31 08:32:23 DEBUG : on/photos/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:32:23 DEBUG : on/photos/: Lookup: name="county029"
2023/03/31 08:32:23 DEBUG : on/photos/: >Lookup: node=on/photos/county029/, err=<nil>
2023/03/31 08:32:23 DEBUG : on/photos/county029/: Attr:
2023/03/31 08:32:23 DEBUG : on/photos/county029/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:32:23 DEBUG : on/photos/county029/: Lookup: name="c029_victoria_lawn_c_id128221.jpg"
2023/03/31 08:32:30 DEBUG : : Statfs:
2023/03/31 08:32:30 DEBUG : : >Statfs: stat={Blocks:276299776 Bfree:208138630 Bavail:208138630 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=<nil>
2023/03/31 08:32:30 DEBUG : /: Attr:
2023/03/31 08:32:30 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:33:14 DEBUG : Google drive root 'website_photos': Checking for changes on remote
2023/03/31 08:34:01 DEBUG : /: Lookup: name="on"
2023/03/31 08:34:01 DEBUG : /: >Lookup: node=on/, err=<nil>
2023/03/31 08:34:01 DEBUG : on/: Attr:
2023/03/31 08:34:01 DEBUG : on/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:34:01 DEBUG : on/: Lookup: name="photos"
2023/03/31 08:34:01 DEBUG : on/: >Lookup: node=on/photos/, err=<nil>
2023/03/31 08:34:01 DEBUG : on/photos/: Attr:
2023/03/31 08:34:01 DEBUG : on/photos/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=<nil>
2023/03/31 08:34:01 DEBUG : on/photos/: Lookup: name="county029"
2023/03/31 08:34:14 DEBUG : Google drive root 'website_photos': Checking for changes on remote

As you can see by the log, Victoria Lawn Image is the one loading forever and not showing. Below that is a refresh of the page and the other pictures doing what they are suppose to be doing.

Any suggestions to my command or how i can resolve this.

Thanks for any assistance

Steve

asdffdsa · March 31, 2023, 12:52pm

do the same files fail each and every time, or do random files fail.

CableGuySteve · March 31, 2023, 1:09pm

It appears at this point the same files fail.
Thanks

asdffdsa · March 31, 2023, 1:12pm

perhaps test without the overhead of the website.
might not make a difference but just one less thing to deal with.

view/download the images on the local machine.

ncw · March 31, 2023, 2:31pm

I'd probably put some caching in here. I don't know whether it will fix your problem, but it will lessen the load on google drive. Here are some suggested parameters - feel free to tweak.

--vfs-cache-mode full
--vfs-cache-max-age 168h
--vfs-cache-max-size 30G

Also this is a good idea

--read-only

asdffdsa · March 31, 2023, 2:59pm

2023/03/31 08:32:23 DEBUG : on/photos/county029/: Lookup: name="c029_victoria_lawn_c_id128221.jpg"
2023/03/31 08:32:30 DEBUG : : Statfs:

in this case, if the delay is from a lookup, would this help?
rclone rc vfs/refresh recursive=true

CableGuySteve · March 31, 2023, 3:19pm

Would i run this as a separate command once the mount is created?

CableGuySteve · March 31, 2023, 3:20pm

Thanks for the information. I am going to work though the suggestions to see how it is effected.

asdffdsa · March 31, 2023, 3:36pm

add the following to the mount command
--rc --rc-no-auth
run the mount command.
make sure the mount is live.
run rclone rc vfs/refresh recursive=true

try that, see if it makes a difference, then we can discuss some additional tweaks.

CableGuySteve · March 31, 2023, 3:56pm

Error Recieved: 2023/03/31 11:54:41 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 11:55:41 Fatal error: mount not ready

Mount Command: rclone mount --daemon --allow-other --read-only --rc --rc-no-auth Photos:website_photos /home/ch/public_html/photos_rclone

asdffdsa · March 31, 2023, 4:05pm

add -vv, run the command again and post full output
try to enclose the text with three backticks, this is the backtick character

Error Recieved: 2023/03/31 11:54:41 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 11:55:41 Fatal error: mount not ready

Mount Command: rclone mount --daemon --allow-other --read-only --rc --rc-no-auth Photos:website_photos /home/ch/public_html/photos_rclone

CableGuySteve · March 31, 2023, 4:10pm

Added -vv as instructed. Only output was this:

2023/03/31 12:07:32 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 12:08:32 Fatal error: mount not ready

Thanks

asdffdsa · March 31, 2023, 4:12pm

from the terminal, copy the command itself and full output and paste it here.
try to enclose it with three backticks so it is formatted like so

2023/03/31 12:07:32 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 12:08:32 Fatal error: mount not ready

CableGuySteve · March 31, 2023, 4:17pm

My Mistake, I reconized my error with -vv as soon as i pasted it. He is the logged information now.

rclone mount --daemon --allow-other --read-only --rc --rc-no-auth Photos:website_photos /home/ch/public_html/photos_rclone -vv
2023/03/31 12:14:47 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "mount" "--daemon" "--allow-other" "--read-only" "--rc" "--rc-no-auth" "Photos:website_photos" "/home/ch/public_html/photos_rclone" "-vv"]
2023/03/31 12:14:47 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 12:14:47 DEBUG : Creating backend with remote "Photos:website_photos"
2023/03/31 12:14:47 DEBUG : Using config file from "/home/ch/.config/rclone/rclone.conf"
2023/03/31 12:14:47 DEBUG : Google drive root 'website_photos': 'root_folder_id = 0AM0NpRiGFpwqUk9PVA' - save this in the config to speed up startup
2023/03/31 12:15:48 DEBUG : Daemon timed out. Terminating daemon pid 9833
2023/03/31 12:15:48 Fatal error: mount not ready

asdffdsa · March 31, 2023, 4:20pm

rclone mount
"When rclone mount is invoked on Unix with --daemon flag, the main rclone program will wait for the background mount to become ready or until the timeout specified by the --daemon-wait flag"

try increasing --daemon-wait

CableGuySteve · March 31, 2023, 4:28pm

So I removed --daemon and this is the results of -vv

[ch@ontar318 rclone]$ rclone mount --allow-other --read-only --rc --rc-no-auth Photos:website_photos /home/ch/public_html/photos_rclone -vv2023/03/31 12:22:49 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "mount" "--allow-other" "--read-only" "--rc" "--rc-no-auth" "Photos:website_photos" "/home/ch/public_html/photos_rclone" "-vv"]
2023/03/31 12:22:49 NOTICE: Serving remote control on http://127.0.0.1:5572/
2023/03/31 12:22:49 DEBUG : Creating backend with remote "Photos:website_photos"
2023/03/31 12:22:49 DEBUG : Using config file from "/home/ch/.config/rclone/rclone.conf"
2023/03/31 12:22:50 DEBUG : Google drive root 'website_photos': 'root_folder_id = 0AM0NpRiGFpwqUk9PVA' - save this in the config to speed up startup
2023/03/31 12:22:50 DEBUG : Google drive root 'website_photos': Mounting on "/home/ch/public_html/photos_rclone"
2023/03/31 12:22:50 DEBUG : : Root:
2023/03/31 12:22:50 DEBUG : : >Root: node=/, err=
2023/03/31 12:23:50 DEBUG : Google drive root 'website_photos': Checking for changes on remote
2023/03/31 12:24:50 DEBUG : Google drive root 'website_photos': Checking for changes on remote
2023/03/31 12:25:50 DEBUG : Google drive root 'website_photos': Checking for changes on remote
2023/03/31 12:26:16 DEBUG : : Statfs:
2023/03/31 12:26:16 DEBUG : : >Statfs: stat={Blocks:276299776 Bfree:208138630 Bavail:208138630 Files:1000000000 Ffree:1000000000 Bsize:4096 Namelen:255 Frsize:4096}, err=
2023/03/31 12:26:16 DEBUG : /: Attr:
2023/03/31 12:26:16 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxr-x, err=
2023/03/31 12:26:50 DEBUG : Google drive root 'website_photos': Checking for changes on remote

asdffdsa · March 31, 2023, 4:29pm

the log looks ok.
now, run the vfs/refresh command from up above and test....

system · April 30, 2023, 4:30pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.