Best mount settings for minimum RAM usage?

Help and Support
1

What is the problem you are having with rclone?

I'm downloading a ton of files directly to an encrypted Google Drive mount and over time it starts using a lot of RAM, around 8GB. The files are never accessed again after they're uploaded so there's no need to keep them in RAM or cache. I'm using the docker image of rclone. Are there better settings for my use case?

Run the command 'rclone version' and share the full output of the command.

rclone v1.60.0

  • os/version: alpine 3.16.2 (64 bit)
  • os/kernel: 5.15.0-1019-oracle (aarch64)
  • os/type: linux
  • os/arch: arm64
  • go/version: go1.19.2
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Encrypted Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

version: "3.9"

services:
    rclone:
        image: rclone/rclone:latest
        container_name: enc
        environment:
            - PUID=1001
            - PGID=1001
        volumes:
            - type: bind
              source: /home/$USER/Enc
              target: /data/enc
              bind:
                propagation: shared
            - /home/$USER/.config/rclone:/root/.config/rclone
            - /etc/passwd:/etc/passwd:ro
            - /etc/group:/etc/group:ro
        devices:
            - /dev/fuse
        cap_add:
            - SYS_ADMIN
        security_opt:
            - apparmor:unconfined
        command: "mount enc: /data/enc --allow-other --allow-non-empty --vfs-cache-mode writes --vfs-cache-max-age 3m --vfs-cache-max-size 2G --vfs-disk-space-total-size 2G --drive-chunk-size 32M --transfers 32 --disable-http2 --drive-pacer-burst 200 --drive-pacer-min-sleep 10ms --dir-cache-time 60s --poll-interval 10s --buffer-size 0 --use-mmap"
        restart: unless-stopped

The rclone config contents with secrets removed.

[drive]
type = drive
client_id = xyz
client_secret = xyz
scope = drive
token = {"access_token":"xyz"}
team_drive = 

[enc]
type = crypt
remote = drive:Enc
filename_encryption = off
directory_name_encryption = false
password = xyz
2

If you have 32 transfers going and that chunk size, you are going to consume some memory.

3

Even after the process is done it still uses a lot of memory and doesn't seem to free it

4

I'm not sure specifically what you changed or what's "a lot".

Can you share some specifics on what was changed / what is the memory usage / log file of what's going on when it is happening?

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.