Can not mount a remote encrypted mount

mega_mount · November 2, 2023, 10:50pm

What is the problem you are having with rclone

I am unable to mount a remote encrypted mount. It runs endlessly

Run the command 'rclone version' and share the full output of the command.

rclone v1.64.2

os/version: ubuntu 22.04 (64 bit)
os/kernel: 5.15.0-88-generic (aarch64)
os/type: linux
os/arch: arm64 (ARMv8 compatible)
go/version: go1.21.3
go/linking: static
go/tags: none

Yes

Which cloud storage system are you using? (eg Google Drive)

Mega

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

/rclone mount crypt: /mnt --vfs-cache-mode full --vfs-cache-max-size 2G --allow-other --allow-root -vv

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

root@ubuntu-8gb-fsn1-1:~# rclone config redacted

[crypt]

type = crypt

remote = mega:

filename_encryption = obfuscate

password = XXX

[crypto]

type = local

[mega]

type = mega

user = XXX

pass = XXX

A log from the command that you were trying to run with the `-vv` flag

root@ubuntu-8gb-fsn1-1:~# rclone mount crypt: /mnt --vfs-cache-mode full --vfs-cache-max-size 2G --allow-other --allow-root -vv

2023/11/02 22:50:11 DEBUG : rclone: Version "v1.64.2" starting with parameters ["rclone" "mount" "crypt:" "/mnt" "--vfs-cache-mode" "full" "--vfs-cache-max-size" "2G" "--allow-other" "--allow-root" "-vv"]

2023/11/02 22:50:11 DEBUG : Creating backend with remote "crypt:"

2023/11/02 22:50:11 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"

2023/11/02 22:50:11 DEBUG : Creating backend with remote "mega:"

2023/11/02 22:50:12 INFO : Encrypted drive 'crypt:': poll-interval is not supported by this remote

2023/11/02 22:50:12 DEBUG : vfs cache: root is "/root/.cache/rclone"

2023/11/02 22:50:12 DEBUG : vfs cache: data root is "/root/.cache/rclone/vfs/crypt"

2023/11/02 22:50:12 DEBUG : vfs cache: metadata root is "/root/.cache/rclone/vfsMeta/crypt"

2023/11/02 22:50:12 DEBUG : Creating backend with remote "/root/.cache/rclone/vfs/crypt/"

2023/11/02 22:50:12 DEBUG : fs cache: renaming cache item "/root/.cache/rclone/vfs/crypt/" to be canonical "/root/.cache/rclone/vfs/crypt"

2023/11/02 22:50:12 DEBUG : Creating backend with remote "/root/.cache/rclone/vfsMeta/crypt/"

2023/11/02 22:50:12 DEBUG : fs cache: renaming cache item "/root/.cache/rclone/vfsMeta/crypt/" to be canonical "/root/.cache/rclone/vfsMeta/crypt"

2023/11/02 22:50:12 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: vfs cache: checking remote fingerprint "6554548925" against cached fingerprint "6554548925"

2023/11/02 22:50:12 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: vfs cache: truncate to size=6554548925 (not needed as size correct)

2023/11/02 22:50:12 DEBUG : vfs cache: looking for range={Pos:0 Size:6554548925} in [{Pos:0 Size:15720448} {Pos:1933127541 Size:139} {Pos:2626527232 Size:77590528}] - present false

2023/11/02 22:50:12 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.RangeSeek from -1 to 15720448 length -1

2023/11/02 22:50:12 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at -1 length 4096 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:12 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.openRange at 15720448 length 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15724544 length 8192 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15732736 length 16384 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15749120 length 32768 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15781888 length 65536 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15847424 length 131072 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:14 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 15978496 length 262144 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:15 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 16240640 length 524288 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:15 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 16764928 length 1048576 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:17 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 17813504 length 1048576 chunkOffset 15720448 chunkSize 134217728

2023/11/02 22:50:17 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 18862080 length 1048576 chunkOffset 15720448 chunkSize 134217728

asdffdsa · November 2, 2023, 10:53pm

welcome to the forum,

that is correct behavior.
if you want to access the files in the remote, the command has to run.
the command will continue to run until you kill it.

that log looks ok, something is accessing and reading the files in the mount.
what is the exact problem you are having?

mega_mount · November 2, 2023, 10:55pm

Thank you for your response, when I run it, I am unable to see any files listed on the directory
I am also unable to see it mounted when I run df -h

asdffdsa · November 2, 2023, 10:56pm

well, looks like some app is reading a file inside the mount as per this log entry
ETRG/Chamber.Of.Secrets.2002.Extended.1080p.BluRay.x264.DTS-ETRG.mkv: ChunkedReader.Read at 18862080 length 1048576

fwiw, i run rclone mount as root user but i never had to use --allow-root

maybe try two tests

using just --allow-other without --allow-root
using just --allow-root without --allow-other

mega_mount · November 3, 2023, 12:02am

Sam for both. I even removed both those flags and tried it, only to get the same result

asdffdsa · November 3, 2023, 12:11am

can you post full output from terminal
rclone tree /mnt --max-depth=1 -vv
and
ls /mnt

mega_mount · November 3, 2023, 12:15am

root@ubuntu-8gb-fsn1-1:~# rclone tree /mnt --max-depth=1 -vv

2023/11/03 00:15:20 DEBUG : rclone: Version "v1.64.2" starting with parameters ["rclone" "tree" "/mnt" "--max-depth=1" "-vv"]

2023/11/03 00:15:20 DEBUG : Creating backend with remote "/mnt"

2023/11/03 00:15:20 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"

2023/11/03 00:15:20 DEBUG : Stat: filePath="/"

2023/11/03 00:15:20 DEBUG : >Stat: fi=, err=

2023/11/03 00:15:20 DEBUG : ReadDir: dir=/

2023/11/03 00:15:20 DEBUG : >ReadDir: names=, err=

/

0 directories, 0 files

2023/11/03 00:15:20 DEBUG : 4 go routines active

root@ubuntu-8gb-fsn1-1:~# ls /mnt
root@ubuntu-8gb-fsn1-1:~#

asdffdsa · November 3, 2023, 12:20am

ok, not sure what is going on, maybe someone else knows for sure.

in the mean time,

maybe need to uncomment
#user_allow_other from /etc/fuse.conf
i would try something other than /mnt
as i run multiple mounts, i use /rclone/mount/name.of.remote
rclone mount crypt: /rclone/mount/crypt --vfs-cache-mode full --vfs-cache-max-size 2G --allow-other -vv
i would try to mount mega:, not crypt:
just a guess, but is the machine a vm running on hetzner?

kapitainsky · November 3, 2023, 7:35am

Your mount seems to be working fine.

But indeed nothing in this folder...

Can you see if your rclone mount is actually running when you run ls?

Run:

ps -ef | grep rclone

and post output here

mega_mount · November 3, 2023, 8:26am

root@ubuntu-8gb-fsn1-1:~# ps -ef | grep rclone
root 860 840 0 Nov02 ? 00:00:03 rclone serve docker -v
root 207712 207677 0 08:26 pts/0 00:00:00 grep --color=auto rclone

mega_mount · November 3, 2023, 8:28am

Yes this is a hetzner VM. Also to clarify, it was working just fine yesterday.
It worked very well for more than a day and I was very pleased with the result. I could finally encrypt my storage box and mount it on my hetzner to serve data.

It is the same command, I changed nothing. then suddenly it stopped working

kapitainsky · November 3, 2023, 8:29am

Does not look that your mount is running so no surprise that there is nothing in /mnt folder.

Run your rclone mount command. Then open another terminal window to inspect /mnt content.

mega_mount · November 3, 2023, 8:34am

Okay, very strange but I rebooted and mounted it again. it worked. Here is the output to examine
root@ubuntu-8gb-fsn1-1:~# rclone mount crypt:/ /mnt -vv

2023/11/03 08:33:01 DEBUG : rclone: Version "v1.64.2" starting with parameters ["rclone" "mount" "crypt:/" "/mnt" "-vv"]

2023/11/03 08:33:01 DEBUG : Creating backend with remote "crypt:/"

2023/11/03 08:33:01 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"

2023/11/03 08:33:01 DEBUG : Creating backend with remote "mega:/"

2023/11/03 08:33:02 DEBUG : fs cache: renaming cache item "mega:/" to be canonical "mega:"

2023/11/03 08:33:02 DEBUG : fs cache: switching user supplied name "mega:/" for canonical name "mega:"

2023/11/03 08:33:02 INFO : Encrypted drive 'crypt:/': poll-interval is not supported by this remote

2023/11/03 08:33:02 NOTICE: Encrypted drive 'crypt:/': --vfs-cache-mode writes or full is recommended for this remote as it can't stream

2023/11/03 08:33:02 DEBUG : Encrypted drive 'crypt:/': Mounting on "/mnt"

2023/11/03 08:33:02 DEBUG : : Root:

2023/11/03 08:33:02 DEBUG : : >Root: node=/, err=

2023/11/03 08:33:24 DEBUG : /: Attr:

2023/11/03 08:33:24 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:26 DEBUG : /: Attr:

2023/11/03 08:33:26 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: Attr:

2023/11/03 08:33:27 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: ReadDirAll:

2023/11/03 08:33:27 DEBUG : test: Skipping undecryptable dir name: not an encrypted file - does not match suffix

2023/11/03 08:33:27 DEBUG : gdp: Skipping undecryptable dir name: not an encrypted file - does not match suffix

2023/11/03 08:33:27 ERROR : : Entry doesn't belong in directory "" (same as directory) - ignoring

2023/11/03 08:33:27 DEBUG : /: >ReadDirAll: item=7, err=

2023/11/03 08:33:27 DEBUG : /: Lookup: name="A Haunting In Venice (2023) [1080p] [WEBRip] [5.1] [YTS.MX]"

2023/11/03 08:33:27 DEBUG : /: >Lookup: node=A Haunting In Venice (2023) [1080p] [WEBRip] [5.1] [YTS.MX]/, err=

2023/11/03 08:33:27 DEBUG : A Haunting In Venice (2023) [1080p] [WEBRip] [5.1] [YTS.MX]/: Attr:

2023/11/03 08:33:27 DEBUG : A Haunting In Venice (2023) [1080p] [WEBRip] [5.1] [YTS.MX]/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: Lookup: name="Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG"

2023/11/03 08:33:27 DEBUG : /: >Lookup: node=Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/, err=

2023/11/03 08:33:27 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/: Attr:

2023/11/03 08:33:27 DEBUG : Harry.Potter.Complete.Collection.2001-2011.1080p.BluRay.x264.DTS-ETRG/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: Lookup: name="The Homecoming (1973) [1080p] [BluRay] [YTS.MX]"

2023/11/03 08:33:27 DEBUG : /: >Lookup: node=The Homecoming (1973) [1080p] [BluRay] [YTS.MX]/, err=

2023/11/03 08:33:27 DEBUG : The Homecoming (1973) [1080p] [BluRay] [YTS.MX]/: Attr:

2023/11/03 08:33:27 DEBUG : The Homecoming (1973) [1080p] [BluRay] [YTS.MX]/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: Lookup: name="The Pink Panther - Complete 11 Film Collection 1963-2009 Eng Subs 720p [H264-mp4]"

2023/11/03 08:33:27 DEBUG : /: >Lookup: node=The Pink Panther - Complete 11 Film Collection 1963-2009 Eng Subs 720p [H264-mp4]/, err=

2023/11/03 08:33:27 DEBUG : The Pink Panther - Complete 11 Film Collection 1963-2009 Eng Subs 720p [H264-mp4]/: Attr:

2023/11/03 08:33:27 DEBUG : The Pink Panther - Complete 11 Film Collection 1963-2009 Eng Subs 720p [H264-mp4]/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

2023/11/03 08:33:27 DEBUG : /: Lookup: name="ff"

2023/11/03 08:33:27 DEBUG : /: >Lookup: node=ff/, err=

2023/11/03 08:33:27 DEBUG : ff/: Attr:

2023/11/03 08:33:27 DEBUG : ff/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=

kapitainsky · November 3, 2023, 8:39am

It is the same output as before - it was always working IMO.

But for whatever reason you were stopping it before using.

rclone mount command must be running for as long as you want to use your mount.

mega_mount · November 3, 2023, 8:42am

No, it is not working when I use any flags, especially when I use cache mode.
without flags it works fine for me. otherwise I keep getting the debug and reading

kapitainsky · November 3, 2023, 8:45am

with --vfs-cache-mode full --vfs-cache-max-size 2G it can take some time for mount to "appear" if you have many cached files which have to be checked against remote. Just wait. Mega is not famous for being fast.

mega_mount · November 3, 2023, 8:50am

Thank you! this makes complete sense. can I clear my cache so it happens faster ?
Also I do not have a ton of free storage on my VPS. Can I set it to clear cache sooner ?
Thank you!

kapitainsky · November 3, 2023, 8:54am

Do you use your mount to write files to remote? If yes then clearing cache can lead to lost files.

But if you use it only for reading then feel free to purge it. It is probably located in /root/.cache/rclone. Just delete it

mega_mount · November 3, 2023, 9:16am

Thank you very much.

I have a few follow up questions, post which I think we can close this thread.

I am using this to write torrents downloads into the crypt. I think it takes time to write the files. If I clear the cache after a few days, I am unlikely to lose or corrupt information yes? since the transfer would have already taken place?
How can I reduce this caching since it takes ages?

kapitainsky · November 3, 2023, 9:22am

I would not do it. Unless you want to monitor cache stats and make sure that all content has been already written.

It only takes time when you restart it. Just wait first time. Then it can run for days or weeks without any delays.

You can try to speed up mega by using their cmd client and connecting rclone to it via webdav - many people report that it is much faster. It was discussed on the forum many times - you can easy find some threads covering it, e.g.: