High memory usage when running `rclone ncdu /path`

Help and Support
1

The rclone mount uses all available memory when I run rclone ncdu /mounted_remote on a mounted filesystem - after scanning about 1.5TB, 6GB of RAM is taken up.

Is this normal behaviour?

Similarly, memory usage increases, only slower when I execute, for example, tar -cf - /mounted_remote > /dev/null

My settings described here: Cryptcheck seems to be downloading files from a remote

edit: Much better after upgrading to the new rclone version. Now rclone mount frees some memory after some time. Unfortunately, still rclone ncdu causes rclone mount memory usage to increase too fast.

2

Running rclone ncdu will cause all the objects to be loaded into memory. I'd guess you have about 6,000,000 objects as rclone usually uses about 1GB memory per 1,000,000 objects. This is normal, but something I'd like to fix.

1 Like
3

Thank you for the explanation. Less but also a lot of memory is eaten by disk operations on a mounted resource. E.g. such simulation tar -cf - /mounted_remote > /dev/null. Is this also normal?

I understand that the recommended way to ex. recover data from remote is rclone copy remote:location /storage?

rclone mount + cp from shell or any other command running on mounted remote would be not recommended to restore large amounts of data?

4

I mean, you can, but it's another layer on top and adds to the time to do something. It's faster/more efficient to not use a mount in that situation.

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.