Rclone, music directory, and plex - Plex eventually stops reading

Help and Support
1

What is the problem you are having with rclone?

I have plex trying to scan a music folder that consists of (at least 3076) folders (organized by Root > artist > album). Plex gets to about 300 artists, and then stalls when scanning. When checking the log files for the mount (not the cache mount) in debug mode:

2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: >Read: read=131072, err=<nil>
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: Read: len=131072, offset=10682368
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: >Read: read=131072, err=<nil>
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: Flush:
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: >Flush: err=<nil>
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: Release:
2020/05/17 10:14:19 DEBUG : Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac: ReadFileHandle.Release closing
2020/05/17 10:14:19 DEBUG : Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac: ChunkedReader.Read at 16773120 length 1048576 chunkOffset 0 chunkSize 134217728
2020/05/17 10:14:19 DEBUG : Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac: ChunkedReader.Read at 17821696 length 1048576 chunkOffset 0 chunkSize 134217728
2020/05/17 10:14:19 DEBUG : &{Various Artists/Mozart/Volume 20 - Litanies, Vespers, etc/Disc 2/15 - Mozart, Wolfgang Amadeus - Dixit et Magnificat K193 2 Magnificat Anima Mea.flac (r)}: >Release: err=<nil>
2020/05/17 10:14:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:14:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:14:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:15:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:15:16 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:15:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:15:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:15:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:16:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:16:16 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:16:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:16:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:16:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:17:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:17:16 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:17:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:17:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:17:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:18:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:18:16 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:18:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:18:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:18:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:19:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:19:16 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:19:31 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:19:46 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:19:53 INFO  : Cleaned the cache: objects 4 (was 4), total size 146 (was 146)
2020/05/17 10:20:01 DEBUG : Google drive root 'Music': Checking for changes on remote
2020/05/17 10:20:16 DEBUG : Google drive root 'Music': Checking for changes on remote

I was trying to look at possibly implementing the cache mount as a way to overcome some of this and see if that would resolve the scanning issues, but, the cache is empty when listing the directory in plex.

rclone.conf:

[gdrivemusic]
type = drive
client_id = 
client_secret = 
scope = drive
token = 
team_drive = 0AF0TKD07fcxXUk9PVA

[gdrivemusic-crypt]
type = crypt
remote = gdrivemusic:Music
filename_encryption = standard
directory_name_encryption = true
password =
password2 =

[gdrivemusic-cache]
type = cache
remote = gdrivemusic-crypt:/
chunk_size = 10M
info_age = 1d

systemd files:

$ cat rclone-gmusic.service
[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/andrew/.config/rclone/rclone.conf
KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount gdrivemusic-crypt: /mnt/rclone/gmusic \
--allow-other \
--buffer-size 256M \
--cache-dir /mnt/plex/.cache/ \
--dir-cache-time 1000h \
--log-level DEBUG \
--log-file /var/log/rclone.log \
--poll-interval 15s \
--timeout 1h \
--umask 002 \
--rc \
--rc-addr 127.0.0.1:5573

ExecStop=/bin/fusermount -uz /mnt/rclone/gmusic
Restart=on-failure
User=andrew
Group=andrew

[Install]
WantedBy=multi-user.target

$ cat rclone-gmusic-cache.service
[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/andrew/.config/rclone/rclone.conf
KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount gdrivemusic-cache: /mnt/rclone/gmusic-cache \
--allow-other \
--buffer-size 256M \
--log-level DEBUG \
--log-file /var/log/rclone.log \
--poll-interval 15s \
--timeout 1h \
--umask 002 \
--rc \
--rc-addr 127.0.0.1:5580

ExecStop=/bin/fusermount -uz /mnt/rclone/gmusic-cache
Restart=on-failure
User=andrew
Group=andrew

[Install]
WantedBy=multi-user.target

Was wondering

What is your rclone version (output from rclone version)

rclone version
rclone v1.51.0
- os/arch: linux/amd64
- go version: go1.13.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04 LTS

Which cloud storage system are you using? (eg Google Drive)

Google Drive

Ultimately, I need help figuring out one of two things:

  1. Why rclone doesn't provide any more logs/Plex keeps stalling, after 300 artists, and what I can do to tweak it to finish.
  2. See if the cache can correct issue 1.
2

You may want to check the Plex logs as looking at the rclone logs, there isn't anything wrong as the mount is still running and checking for updates.

With small music files, using a large buffer size wouldn't provide much. Doesn't seem like you were running out of memory on the logs.

3

Unfortunately, the plex logs are not helpful (and not currently working, something else I am trying to debug). But ultimately, when they were working, I would just see it stall or move incredibly slowly. I attributed it to rclone's drive listing having issues with it staying in cache (if the scan is taking longer than an hour to run, it would eventually time out the cache).

I mainly using the buffer size for larger flac files.

4

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.