Rclone mount, chewing up all my ram

Suspected Bug
1

What is the problem you are having with rclone?

Chewing up all my ram, RAM just continously rises until system is unreachable. Than rclone mount terminates.

What is your rclone version (output from rclone version)

rclone v1.42

  • go version: go1.10.1

Which OS you are using and how many bits (eg Windows 7, 64 bit)

  • os/arch: linux/amd64

Which cloud storage system are you using? (eg Google Drive)

Google Drive(encrypted)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

/etc/systemd/system/rclone.service

[Unit]
Description=Google Drive (rclone)

[Service]
Type=simple
User=dragun
Group=dragun
ExecStart=/usr/bin/rclone mount
--config=/home/dragun/.config/rclone/rclone.conf
--allow-other
--buffer-size 500M
--dir-cache-time 72h
--drive-chunk-size 32M
--fast-list
--vfs-cache-mode writes
--vfs-read-chunk-size 128M
--vfs-read-chunk-size-limit off
--stats 1m
--log-level DEBUG
--log-file=/home/dragun/rclone.log gcrypt:/ /gfuck
ExecStop=/bin/fusermount -u /gfuck
Restart=always
RestartSec=10

[Install]
WantedBy=default.target

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

2019/11/12 21:19:32 DEBUG : rclone: Version "v1.42" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/dragun/.config/rclone/rclone.conf" "--allow-other" "--buffer-size" "500M" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--fast-list" "--vfs-cache-mode" "writes" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "--stats" "1m" "--log-level" "DEBUG" "--log-file=/home/dragun/rclone.log" "gcrypt:/" "/gfuck"]
2019/11/12 21:19:32 DEBUG : Using config file from "/home/dragun/.config/rclone/rclone.conf"
2019/11/12 21:19:32 Fatal error: Can not open: /gfuck: open /gfuck: transport endpoint is not connected
2019/11/12 21:19:42 DEBUG : rclone: Version "v1.42" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/dragun/.config/rclone/rclone.conf" "--allow-other" "--buffer-size" "500M" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--fast-list" "--vfs-cache-mode" "writes" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "--stats" "1m" "--log-level" "DEBUG" "--log-file=/home/dragun/rclone.log" "gcrypt:/" "/gfuck"]
2019/11/12 21:19:42 DEBUG : Using config file from "/home/dragun/.config/rclone/rclone.conf"
2019/11/12 21:19:42 Fatal error: Can not open: /gfuck: open /gfuck: transport endpoint is not connected
2019/11/12 21:19:52 DEBUG : rclone: Version "v1.42" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/dragun/.config/rclone/rclone.conf" "--allow-other" "--buffer-size" "500M" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--fast-list" "--vfs-cache-mode" "writes" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "--stats" "1m" "--log-level" "DEBUG" "--log-file=/home/dragun/rclone.log" "gcrypt:/" "/gfuck"]
2019/11/12 21:19:52 DEBUG : Using config file from "/home/dragun/.config/rclone/rclone.conf"
2019/11/12 21:19:52 Fatal error: Can not open: /gfuck: open /gfuck: transport endpoint is not connected
2019/11/12 21:20:02 DEBUG : rclone: Version "v1.42" starting with parameters ["/usr/bin/rclone" "mount" "--config=/home/dragun/.config/rclone/rclone.conf" "--allow-other" "--buffer-size" "500M" "--dir-cache-time" "72h" "--drive-chunk-size" "32M" "--fast-list" "--vfs-cache-mode" "writes" "--vfs-read-chunk-size" "128M" "--vfs-read-chunk-size-limit" "off" "--stats" "1m" "--log-level" "DEBUG" "--log-file=/home/dragun/rclone.log" "gcrypt:/" "/gfuck"]
2019/11/12 21:20:02 DEBUG : Using config file from "/home/dragun/.config/rclone/rclone.conf"
2019/11/12 21:20:02 DEBUG : Encrypted drive 'gcrypt:/': Mounting on "/gfuck"
2019/11/12 21:20:02 DEBUG : vfs cache root is "/home/dragun/.cache/rclone/vfs/gcrypt"
2019/11/12 21:20:02 DEBUG : Adding path "vfs/forget" to remote control registry
2019/11/12 21:20:02 DEBUG : : Root:
2019/11/12 21:20:02 DEBUG : : >Root: node=/, err=
2019/11/12 21:20:03 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:03 DEBUG : Google drive root '': Checking for changes on remote
2019/11/12 21:20:03 DEBUG : Google drive root '': All changes were processed. Waiting for more.
2019/11/12 21:20:03 DEBUG : BakenStreams.ca: No export formats found for "application/vnd.google-apps.site"
2019/11/12 21:20:03 DEBUG : Untitled document.docx: Skipping undecryptable file name: illegal base32 data at input byte 8
2019/11/12 21:20:03 DEBUG : Make a Request (Responses).xlsx: Skipping undecryptable file name: illegal base32 data at input byte 4
2019/11/12 21:20:03 DEBUG : trans-banner.png: Skipping undecryptable file name: illegal base32 data at input byte 5
2019/11/12 21:20:03 DEBUG : trans-banner2.png: Skipping undecryptable file name: illegal base32 data at input byte 5
2019/11/12 21:20:03 DEBUG : SigSupport.docx: Skipping undecryptable file name: illegal base32 data at input byte 10
2019/11/12 21:20:03 DEBUG : requsts.jpg: Skipping undecryptable file name: illegal base32 data at input byte 7
2019/11/12 21:20:03 DEBUG : Untitled document.docx: Skipping undecryptable file name: illegal base32 data at input byte 8
2019/11/12 21:20:03 DEBUG : Baken.PNG: Skipping undecryptable file name: illegal base32 data at input byte 5
2019/11/12 21:20:03 DEBUG : Getting started: Skipping undecryptable file name: illegal base32 data at input byte 7
2019/11/12 21:20:03 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:03 DEBUG : TestTV/: Attr:
2019/11/12 21:20:03 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:03 DEBUG : TestTV/: Lookup: name="Criminal Minds"
2019/11/12 21:20:05 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:05 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:05 DEBUG : TestTV/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:05 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:05 DEBUG : TestTV/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/: Lookup: name="Criminal Minds"
2019/11/12 21:20:05 DEBUG : TestTV/: >Lookup: node=TestTV/Criminal Minds/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/: >Lookup: node=TestTV/Criminal Minds/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: Lookup: name="Season 5"
2019/11/12 21:20:05 DEBUG : TestTV/: Lookup: name="Frasier"
2019/11/12 21:20:05 DEBUG : TestTV/: >Lookup: node=TestTV/Frasier/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: Lookup: name="Season 5"
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/: Lookup: name="Season 5"
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/: >Lookup: node=TestTV/Frasier/Season 5/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/Season 5/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/Season 5/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/Frasier/Season 5/: Lookup: name="Frasier - S05E14 - The Ski Lodge SDTV.mp4"
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: >Lookup: node=TestTV/Criminal Minds/Season 5/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/: >Lookup: node=TestTV/Criminal Minds/Season 5/, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: Attr:
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: Lookup: name="Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv"
2019/11/12 21:20:05 DEBUG : TestTV/Criminal Minds/Season 5/: Lookup: name="Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv"
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: >Lookup: node=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: >Lookup: node=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Attr: a=valid=1s ino=0 size=509603984 mode=-rw-r--r--, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Attr: a=valid=1s ino=0 size=509603984 mode=-rw-r--r--, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : /: Lookup: name="TestTV"
2019/11/12 21:20:06 DEBUG : /: >Lookup: node=TestTV/, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : TestTV/: Lookup: name="Criminal Minds"
2019/11/12 21:20:06 DEBUG : TestTV/: >Lookup: node=TestTV/Criminal Minds/, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/: Lookup: name="Season 5"
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/: >Lookup: node=TestTV/Criminal Minds/Season 5/, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: Lookup: name="Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv"
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/: >Lookup: node=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Attr:
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Attr: a=valid=1s ino=0 size=509603984 mode=-rw-r--r--, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=OpenReadOnly
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=OpenReadOnly
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=O_RDONLY
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=O_RDONLY
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fd=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r), err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fd=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r), err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fh=&{TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r)}, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fh=&{TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r)}, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=OpenReadOnly
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=OpenReadOnly
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=O_RDONLY
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fd=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r), err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fh=&{TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r)}, err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=O_RDONLY
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: >Open: fd=TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv (r), err=
2019/11/12 21:20:06 DEBUG : TestTV/Criminal Minds/Season 5/Criminal Minds - S05E21 - Exit Wounds HDTV-720p.mkv: Open: flags=OpenReadOnly

The rest of most of the log:
https://hastebin.com/otiqumequv.sql

2

You are using 500M per file opened so that's chewing up your memory.

1.42 is pretty old too so I'd update that.

3

I am confident that 500M isn't the issue as I have had it running at 1G for the last 3 months without issue. Also there were only 5 streams happening when RAM was all used up. This has only popped up in the last 2 days. I mean I can try updating but not sure why all of the sudden this starts happening.

4

You are also mixing encrypted files with regular ones too.

1 Like
5

Is that not a good thing? I guess it';s atleast an uneeded distraction. I have set --buffer-size to 128M, updated to 1.50.1. So far so good but it's only been 10 minutes with little activity.

Appreciate the response!

6

I mean, technically, it doesn't break anything but just errors out as they are unneeded errors. Usually you don't mix them as it's just noise in your log files.

1 Like
7

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.