Rclone not clearing chunks automatically

Help and Support
1

Hi,

It seems that rclone is not clearing chunks automatically from the temp cache folder.

rclone.conf

[media_cache]
type = cache
remote = media:
plex_url = http://127.0.0.1:32400
plex_username = [REDACTED]
plex_password = *** ENCRYPTED ***
chunk_size = 5M
info_age = 1h0m0s
chunk_total_size = 500M
chunk_path = /dev/shm
plex_token = [REDACTED]
chunk_no_memory = true
chunk_clean_interval = 30s
workers = 5

media.service (systemd unit file)

[Unit]
After=network-online.target

[Service]
Type=notify
ExecStart=/usr/bin/rclone mount \
        --config=/home/pi/.config/rclone/rclone.conf \
        --read-only \
        --allow-other \
#       --vfs-cache-mode minimal \
#       --dir-cache-time 24h \
#       --buffer-size 256M \
        -v --log-file /opt/media.log \
        media_cache:/ /mnt/media
ExecStop=/bin/fusermount -u /mnt/media
User=pi
Group=pi

[Install]
WantedBy=multi-user.target

After at least 10 minutes of stopping Plex playback:

ncdu 1.13 ~ Use the arrow keys to navigate, press ? for help                    
--- /dev/shm -------------------------------------------------------------------
  953.4 MiB [##########] /media_cache
2

What rclone version are you running?
Can you share a log file with -vv of the mount?

3 
rclone version
rclone v1.50.2
- os/arch: linux/arm
- go version: go1.13.4

This is with -v. If that's not sufficient, then I'm happy to provide -vv logs.

4

It doesn't seem from the logs you are using that much so perhaps you have something else in there?

2020/01/09 23:17:23 INFO  : cache-cleanup: chunks 7, est. size: 70M
2020/01/09 23:18:23 INFO  : cache-cleanup: chunks 3, est. size: 30M
2020/01/09 23:19:24 INFO  : cache-cleanup: chunks 1, est. size: 10M
2020/01/09 23:20:24 INFO  : cache-cleanup: chunks 1, est. size: 10M
2020/01/09 23:21:24 INFO  : cache-cleanup: chunks 1, est. size: 10M
2020/01/09 23:25:24 INFO  : cache-cleanup: chunks 10, est. size: 90.455M
2020/01/09 23:26:24 INFO  : cache-cleanup: chunks 10, est. size: 90.455M
2020/01/09 23:27:24 INFO  : cache-cleanup: chunks 6, est. size: 50.455M
2020/01/09 23:28:24 INFO  : cache-cleanup: chunks 8, est. size: 70.455M
2020/01/09 23:29:24 INFO  : cache-cleanup: chunks 7, est. size: 60.455M
2020/01/09 23:30:24 INFO  : cache-cleanup: chunks 8, est. size: 70.455M
2020/01/09 23:31:24 INFO  : cache-cleanup: chunks 5, est. size: 40.455M
2020/01/09 23:32:24 INFO  : cache-cleanup: chunks 4, est. size: 30.455M
2020/01/09 23:39:24 INFO  : cache-cleanup: chunks 10, est. size: 93.416M
2020/01/09 23:40:28 INFO  : cache-cleanup: chunks 35, est. size: 343.416M
5

Nope, its from the rclone cache.

6

Can you share the output of the directory?

7

I just rebooted it and since I was caching it to /dev/shm, the RAM was flushed, but I can tell you that the name of the content I was playing appeared as a folder with numeric file names.

8

If you check the log messages and look for the last

2020/01/09 23:40:28 INFO  : cache-cleanup: chunks 35, est. size: 343.416M

That should match up with the disk space consumed in your cache area.

9

I will test for this issue again. Basically, it should clear the cache after Plex playback stops and chunk_clean_interval passes, right?

Or just chunk_clean_interval from the time the chunks are downloaded?

10

It cleans the chunks based on the chunk interval and it being over the chunk total size value:

The total size that the chunks can take up on the local disk.

If the cache exceeds this value then it will start to delete the oldest chunks until it goes under this value.
11

Well clearly it isn't doing that. I have files in the folder from Plex playback several hours ago.

The more important issue that's plaguing me is that Plex seems to scan folders a lot slower with cache. Without it, it would zip through with more than 20 folders a second. Now it takes 5-6secs per folder.

I'm not noticing faster playback or faster seeks either with the supposed Plex integration proving to be a gimmick only.

12

I'm happy to help but you haven't provided too much to help yet other than it doesn't work.

The cache with plex integration is slower as that's the whole purpose of it. It slows down other API related tasks to 1 worker except if you are playing movies. Once a movie is picked up, it increases the workers.

You wouldn't see faster playback or seeks really as that's not the purpose. The purpose of the cache is to handle keeping chunks on disk as it's helpful if you have a player that opens and closes files a lot. It also reduces API hits.

If you want to provide some of the details to help see what the issue is, we're happy to help.

13

Well, what details do you need? I've started using most of your rclone.service unit file. Do you get fast seeks with Plex?

14

Recreate the issue with debug log and share the contents of the cache directory.

My use case is watching TV and Movies so it's rare to seek around.

15

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.