Rclone mount crashed, because it is out of memory (v1.15.3 | Linux | Google Drive)

Help and Support
1

What is the problem you are having with rclone?

I have the problem, that the rclone mount crashes when updating a Plex tv-library.

For a lone time I got the same error as postet below ("normal" start), but thought this is no bigger problem, because there was no issue for a year or so. But today I updated my tv-library on my Google-Drive with alot of anime in it (One show could contain single folders with >100 video files) and the mount crashed with the "out of memory" error. I tried to allocate more memory to it (from 20M to 50M) and installed the newes 'go' but I got the same error message.

If I unmount the Google-Drive (fusermount -u /mnt/gsuite) and mount it again i can use Plex as usual, till I update the tv-library where it run in the same error again and again.

Since I do not know how to solve this problem I hope you guys could help me.

What is your rclone version

rclone v1.53.3

  • os/arch: linux/386
  • go version: go1.15.5

Which OS you are using and how many bits

Proxmox VE 6.3-2 ~ Linux (64bit)

Rclone is running directly on the Proxmox installation
Plex is running in a Proxmox-Container (other IP)

Which cloud storage system are you using?

Google Drive (G Suite)

The command you were trying to run

rclone mount --allow-other --allow-non-empty gcrypt: /mnt/gsuite

The rclone config contents with secrets removed.

[gcache]
type = cache
remote = gsuite:/crypt
plex_url = https://192.168.188.102:32400/
plex_username = XXX
plex_password = *** ENCRYPTED ***
chunk_size = 50M
info_age = 1d
chunk_total_size = 50G
plex_token = XXX

[gcrypt]
type = crypt
remote = gcache:/media
filename_encryption = off
directory_name_encryption = false
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***

[gcrypt2]
type = crypt
remote = gsuite:/crypt
filename_encryption = off
directory_name_encryption = false
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***

[gsuite]
type = drive
scope = drive
token = XXX
client_id = XXX
client_secret = XXX

A "normal" start, mounting with rclone with no problems using Plex

root@pve:~# rclone mount --allow-other --allow-non-empty gcrypt: /mnt/gsuite &
[1] 22895
root@pve:~# 2020/12/04 13:06:20 ERROR : plex: websocket.Dial wss://192.168.188.102:32400/:/websockets/notifications?X-Plex-Token=XXX: x509: cannot validate certificate for 192.168.188.102 because it doesn't contain any IP SANs

A log from the command with the -v flag

root@pve:~# 2020/12/04 12:57:55 ERROR : plex: websocket.Dial wss://192.168.188.102:32400/:/websockets/notifications?X-Plex-Token=XXX: x509: cannot validate certificate for 192.168.188.102 because it doesn't contain any IP SANs
2020/12/04 12:57:55 INFO  : gcache: Cache DB path: /root/.cache/rclone/cache-backend/gcache.db
2020/12/04 12:57:55 INFO  : gcache: Cache chunk path: /root/.cache/rclone/cache-backend/gcache
2020/12/04 12:57:55 INFO  : gcache: Chunk Memory: true
2020/12/04 12:57:55 INFO  : gcache: Chunk Size: 50M
2020/12/04 12:57:55 INFO  : gcache: Chunk Total Size: 50G
2020/12/04 12:57:55 INFO  : gcache: Chunk Clean Interval: 1m0s
2020/12/04 12:57:55 INFO  : gcache: Workers: 4
2020/12/04 12:57:55 INFO  : gcache: File Age: 1d

A log from the command with the -vv flag

root@pve:~# 2020/12/04 12:59:46 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "mount" "-vv" "--allow-other" "--allow-non-empty" "gcrypt:" "/mnt/gsuite"]
2020/12/04 12:59:46 DEBUG : Creating backend with remote "gcrypt:"
2020/12/04 12:59:46 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2020/12/04 12:59:46 DEBUG : Creating backend with remote "gcache:/media"
2020/12/04 12:59:46 DEBUG : Creating backend with remote "gsuite:/crypt/media"
2020/12/04 12:59:46 DEBUG : Google drive root 'crypt/media': root_folder_id = "XXX" - save this in the config to speed up startup
2020/12/04 12:59:47 DEBUG : fs cache: renaming cache item "gsuite:/crypt/media" to be canonical "gsuite:crypt/media"
2020/12/04 12:59:47 DEBUG : gcache: wrapped gsuite:crypt/media at root media
2020/12/04 12:59:47 ERROR : plex: websocket.Dial wss://192.168.188.102:32400/:/websockets/notifications?X-Plex-Token=XXX: x509: cannot validate certificate for 192.168.188.102 because it doesn't contain any IP SANs
2020/12/04 12:59:47 INFO  : gcache: Cache DB path: /root/.cache/rclone/cache-backend/gcache.db
2020/12/04 12:59:47 INFO  : gcache: Cache chunk path: /root/.cache/rclone/cache-backend/gcache
2020/12/04 12:59:48 ERROR : /root/.cache/rclone/cache-backend/gcache.db: Error opening storage cache. Is there another rclone running on the same remote? failed to open a cache connection to "/root/.cache/rclone/cache-backend/gcache.db": timeout
2020/12/04 12:59:48 Failed to create file system for "gcrypt:": failed to make remote "gcache:/media" to wrap: failed to start cache db: failed to open a cache connection to "/root/.cache/rclone/cache-backend/gcache.db": timeout

When the mount crashes, which cause the problems

[...] 
2020/12/04 11:42:47 ERROR : media/Animation/One Piece (1998)/Season 15/One Piece - S15E35.mp4.bin: failed to authenticate to Plex: failed to obtain token: EOF
runtime: out of memory: cannot allocate 54525952-byte block (3336732672 in use)
fatal error: out of memory
runtime: out of memory: cannot allocate 54525952-byte block (3336732672 in use)
fatal error: out of memory

runtime stack:
runtime.throw(0x92502bd, 0xd)
        runtime/panic.go:1116 +0x6a
runtime.largeAlloc(0x3200200, 0x9040101, 0x9042240)
        runtime/malloc.go:1179 +0x15d
runtime.mallocgc.func1()
        runtime/malloc.go:1071 +0x39
runtime.systemstack(0xbb801c0)
        runtime/asm_386.s:391 +0x53
runtime.mstart()
        runtime/proc.go:1116

Full-Error:

Greetings, Clone4Ever

2

You ran out of resources so you'd need to add more memory.

The cache backend is deprecated and unsupported as well. You should look into vfs-cache-mode full which replaces it.

1 Like
3

So which memory do you refer to (Chunk Size, Chunk Total Size, or other one)?

And what do you mean with 'The cache backend is deprecated and unsupported as well', what do I have to add/ delete/ remove to make it supported again? Just adding the line"--vfs-cache-mode full" to to my mount command?

4

None of those. The memory in container I'd imagine is too low.

See this: https://rclone.org/cache/

See this as you'd remove the cache backend and just use the crypt remote:

1 Like
5

That would be strange, since the system has 64GB of wich Plex takes 6GB (tested with 12GB > same error) which should easily suffice (?).

Okay, if I have understood correctly I should remove the cache option from my rclone config, and add the vfs-cache like this to the mount command. Did I understand you correctly?

rclone mount --allow-other --allow-non-empty --vfs-cache-mode full gcrypt: /mnt/gsuite &

ext4 as filesystem should be any problem?

6

The error is out of memory so I'd check your container config as it's not your system memory but whatever you setup for the container.

That should not be used.

The rest looks good.

Most folks with Google Drive use a large dir cache time since it's a polling remote.

I use:

 --dir-cache-time 1000h

You probably want to tweak the location and size/times based on your needs:

--cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 500G --vfs-cache-max-age 336h

As that's my example that I use.

1 Like
7

Thank you, I tried a bit with the config and come up with the following which is working so far (for the last ~30min.).

Deleting the cache in the rclone config was no good idea. When deleting die "old" cache the problem was that the other configs would not work anymore because the remote was still the cache. Even changing the remote path to the "new" cache path (I used the same as you do /cache) was not working.

So I set up the cache in the rclone config again and started the mount as follows:

rclone mount --allow-other --cache-dir=/cache --dir-cache-time 168h --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 72h gcrypt: /mnt/gsuite

When scanning the library now I got the following errors

2020/12/04 17:29:26 ERROR : plex: websocket.Dial wss://192.168.188.102:32400/:/websockets/notifications?X-Plex-Token=XXX: x509: cannot validate certificate for 192.168.188.102 because it doesn't contain any IP SANs

2020/12/04 17:39:07 ERROR : media/Animation/One Piece (1998)/Season 11/One Piece - S11E12.avi.bin: failed to authenticate to Plex: failed to obtain token: EOF

2020/12/04 17:46:35 ERROR : media/Animation/One Piece (1998)/Season 12/One Piece - S12E06.mkv.bin: unexpected conditions during reading. current position: 241362496, current chunk position: 209715200, current chunk size: 17600512, offset: 31647296, chunk size: 50M, file size: 241478700

From my point of view I think that I messed up some settings, and for now it is just a bypass to make it run but no solution. :confused:

8

you have two remotes
gcrypt uses cache backend
gcrypt2 does not use the cache backend.

so for gcrypt, should be easy to not use the cache backend.
this should do it.
remote = gcache:/media
to
remote = gsuite:crypt/media

if you are using the same password and password2 for both gcrypt and gcrypt2 then you might not need both of those remotes, as they point ot the same remote+path of gsuite:crypt

1 Like
9

So after a day I can say that the "double" cache was a bad idea and the same problem as before occurred. Now I deleted the gcache again and changed the gcrypt as mentioned by @asdffdsa.

Since Plex was so nice to delete all the seasons in the library again (damn!) I try it with this settings. Also I set up more RAM to the Proxmox-Container, hopefully with less memory problems :roll_eyes:.

Any Idea how to solve the problem with the Errors mentioned yesterday?

2020/12/04 17:29:26 ERROR : plex: websocket.Dial wss://192.168.188.102:32400/:/websockets/notifications?X-Plex-Token=XXX: x509: cannot validate certificate for 192.168.188.102 because it doesn't contain any IP SANs

2020/12/04 17:39:07 ERROR : media/Animation/One Piece (1998)/Season 11/One Piece - S11E12.avi.bin: failed to authenticate to Plex: failed to obtain token: EOF

2020/12/04 17:46:35 ERROR : media/Animation/One Piece (1998)/Season 12/One Piece - S12E06.mkv.bin: unexpected conditions during reading. current position: 241362496, current chunk position: 209715200, current chunk size: 17600512, offset: 31647296, chunk size: 50M, file size: 241478700

Thanks :slight_smile:

10

have you tried to search the forum?

Cannot validate certificate for Owncloud

1 Like
11

Based on the os/arch he runs the 32 bit process
Out of memory occurs on 3336732672 bytes which is ~3.3GB of memory. Linux has a 4GB memory limit with 32 bit so his 64GB is nearly useless with 32 bit processes
Loading up a 64 bit rclone process should fix the out of memory issue imo

3 Likes
12

Ah, as in download the 64 bit version of rclone rather than the 32 bit version. Yes that should help a lot :slight_smile:

1 Like
14

It look like updating rclone to the 64bit version lead to the solution (do not even know why I had the 32bit installed).

I use the following code to mount my Google Drive with rclone.

rclone mount --allow-other --no-check-certificate --cache-dir=/cache --dir-cache-time 168h --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 72h gcrypt: /mnt/gsuite &

For the last days there was no problem at all, so it seams to work as it should :).

Thank all of you guys.

1 Like
15

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.