What is the problem you are having with rclone?
I have tested this on more than one machine and more than one distribution with the same end-results so it's definitely a common problem and not machine-specific.
I have been monitoring an rclone mount (see the screenshots) over a number of hours and days as I found that it always starts using the same amount of memory (about 137K) when initially loading and then steadily keeps using more memory over the monitored timeframe.
This happens with or without any usage of the mount during the same timeframe. In the instance shown in the screenshots the machine was booted and other apps were used but the mount was never accessed at all.
The RAM usage can easily grow in size to over 4GB after a day or two of uptime.
I recently changed the log-level from info to debug to see if I can ascertain what was happening with the mount during this time and this has shown that even without the mount being used or accessed since bootup its still actively busy enumerating all the folders in the mounted GDrive. It looks like its busy scanning the full directory tree. This is a 48TB drive with millions of files.
So what I'm now starting to think is that perhaps there is no memory leak but rather as the folder enumeration continues its using systematically more memory to store the scanned information. Can this be the case?
My understanding is that rclone accesses and fetches info from GDrive only when you are actually traversing a particular directory tree and that it only fetches enough info into the local cache to satisfy the current browsing requests? Am I wrong? How do I prevent the automatic folders enumeration?
I am essentially trying to have a readily available GDrive mount that will only retrieve what is being used at a particular point in time and then free those resources as soon as they are no longer actively being used by any application. Basically like Google File Stream for Linux.
Thanks for any pointers
What is your rclone version (output from
Which OS you are using and how many bits (eg Windows 7, 64 bit)
KDE Neon 18.04 64 bit, Proxmox 6.2-4 (Debian)
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
/usr/bin/rclone mount --config=/(omitted)/rclone.conf --allow-other --checksum --buffer-size 30M --use-mmap --fast-list --cache-dir /tmp --vfs-cache-mode writes --drive-chunk-size 16M --attr-timeout 30s --drive-export-formats link.html --drive-use-trash=true --drive-alternate-export=true --drive-acknowledge-abuse=true --log-level DEBUG --syslog gdrive: (/local mountpoint)
The rclone config contents with secrets removed.
[gdrive] type = drive client_id = xxxx client_secret = xxxx scope = drive root_folder_id = xxxx token = xxxx
A log from the command with the
Paste log here