General advice on how to optimize rclone for playing music files

[ioorclone]

What is the problem you are having with rclone?

I am seeking general advice on mount or other settings to optimize for streaming music files. Currently it takes an average of 8-10 seconds to start playing a file. Ideally I would like to get this time to 1-2 seconds.

Version

rclone v1.56.0
- os/version: debian bullseye/sid (64 bit)
- os/kernel: 5.11.0-34-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.5
- go/linking: static
- go/tags: none

The command you were trying to run

I have tried many different configurations. Specifically adjusting the following settings:

1. --vfs-cache-mode off, minimal, writes, full

Result: With the vfs cache off/minimal/writes there is a slight shudder when the file starts playing. This studded can be eliminated by using the full mode.
None of these other modes decrease the time to start playing.

2. Priming the dir-cache with this setting:

ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 127.0.0.1:5572 _async=true

Result: After running this command, there is a big improvement in the time find . | wc -l but it doesn't seem to improve time to play.

3. Adjusting the following settings from very high values to very low values:

--buffer-size
--max-read-ahead
--vfs-read-chunk-size
--vfs-read-chunk-size-limit
--transfers

Result: If I have each one of these settings at a very high value, very low value or the default I do not see any difference in the time to start playing the file.

4. I have tried wrapping my remote in the depreciated cache backend because in other threads it was mentioned that vfs is not suitable for music files.

Result: This actually added 1-2 seconds to the time to play. I adjusted the chunk size settings form high to low, which did not have a noticeable effect.

5. I have tried two different cloud provider back ends.

Result: The results are similar for both. I do not believe that bandwidth is playing a factor here.

I have not come upon any settings that seem to make a measurable difference. Are there other settings I can try or that you would advise I concentrate on?

Does anyone use rclone for music playback?

[asdffdsa (jojothehumanmonkey)]

hi,

as you did not answer most of the help and support questions, hard to give a good answer.

rclone rc vfs/refresh
that is for the vfs dir cache, not the vfs file cache, so does not affect time to start play.

how are you streaming, using a media server like plex or using a media player like vlc?

there are a lot of things that can affect start time, not related to rclone.
--- your internet connection

[ioorclone]

Hi, thanks for the reply.

@asdffdsa

how are you streaming, using a media server like plex or using a media player like vlc?

I am playing the file directly from a rclone mount with a music player. There is no servers or transcoding involved.

@asdffdsa

there are a lot of things that can affect start time, not related to rclone.
--- your internet connection
--- could be an issue with a weak cpu transcoding.

I believe the internet connection is sufficient: 30Mb/s down. These are small .mp3 files average around 7MB. I don't suspect that bandwidth is the bottleneck. For example, I just did a test and downloaded a 6mb file using the mount:
cp "/mount/file.mp3 /tmp
In less than 6 seconds I had copied the entire file from the rclone mount locally.

I did not post a specific configuration because I have tried probable 50 variations of setting combinations. Honestly nothing I have tried has made any noticeable difference apart from:

rclone mount
--fast-list
--vfs-cache-mode=full
remote: /path

which would use all the default settings.

The only setting I can adjust that makes a difference is --vfs-cache-mode. If I set it to anything less than full I get shuddering within the first few seconds. Setting it to 'full' fixes the shuddering issue but it does not decrease the time to start playing.

If I could even get the time to play down to 3-4 seconds it would be acceptable for a music player application but 8-10 seconds is longer than the time it takes to download the entire file.

Here are some recent threads I was researching to try to find possible solutions:

• Status about using Rclone for music storage / playback in 2021. Access times improved?
• Testing for new vfs cache mode features - #10 by Rootax
• Rclone Cache Is Better/Faster for Plex Than VFS - #14 by Animosity022
• Rclone Mount slow read speed - #22 by Nicolas_Girard

I have tried all of the suggestions in those threads. Is there anything more I could post that would help analyze the issue or is it possible that 8-10 seconds is the best performance I can expect for playing a music file from an rclone mount.

[asdffdsa (jojothehumanmonkey)]

@ioorclone

anything more I could post that would help analyze the issue

--- output of rclone version
--- redacted config file
--- the best performing command and its debug log when trying to stream a media file.

[ioorclone]

@asdffdsa

well, i already shared that but you are not willing to provide it.

Sorry to give that impression. I wanted to get some general ideas first.
Here are the missing parts including a config and log with default settings.

Which cloud storage system are you using? (eg Google Drive)

-koofr
-also tried same tests with pcloud, similar results.

Bandwidth

Speedtest:
- Server close to koofr data center
- 76ms ping
- Download: 22.8
- Upload: 5.77

The command you were trying to run (eg rclone copy /tmp remote:tmp)

[Unit]
Description=rclone mount
AssertPathIsDirectory=/path

[Service]
Type=notify
ExecStart=/usr/bin/rclone mount \
    -vv \
    --fast-list \
    --cache-dir=/tmp/rclone \
    --vfs-cache-mode=full \
crypt: /mount
ExecStop=/bin/fusermount -u /mount
Restart=always
RestartSec=10

[Install]
WantedBy=default.target

The rclone config contents with secrets removed.

[remote]
type = koofr
user = xxx
password = xxx

[crypt]
type = crypt
remote = remote:
password = xxx
password2 = xxx

A log from the command with the -vv flag

rclone.log (23.6 KB)

Test results

The player app was already loaded into memory with a playlist containing a single 3mb file "file.mp3".
Result using mount: 10 seconds from pressing start button to when the file started playing.

Same file copied entirely without using the mount:
$ rclone copy "remote:path/file.mp3" /tmp --progress
Result: roughly 2-3 seconds

This test was with a 3mb file. I can repeat the test with a file 5x the size (15mb) and get the same results, roughly 8-10 seconds on average to start playing the file.

Any suggestions for adjustments I can try to the --vfs-cache flags or other mount configs to decrease the time to start playing the file?

[asdffdsa (jojothehumanmonkey)]

--- --fast-list does nothing on a mount
--- --cache-db-purge does nothing, can remove that.
--- there is something wrong with that debug log, missing the timestamps.

[ioorclone]

Interesting you mention that, I have also noticed that timestamps would be useful when looking at these -vv logs.

It might be because I am mounting as a Linux systemd service and I capture the logs like this:

systemd mount.service file:

[Unit]
...

[Service]
...

StandardOutput=append:/logs/mount.log
StandardError=append:/logs/error.log

[Install]

Aside from the missing timestamps, does anything else stand out as a reason for the long start to play time?

[asdffdsa (jojothehumanmonkey)]

--- imho, for testing, run the rclone mount from the command line.
--- make sure the vfs file cache is empty before running the command
--- i am not an expert with rclone mount log, but the timestamps should be helpful so use a rclone debug log.

[ioorclone]

Here is a result using the same file and config as the previous test. This time using --log-file and -vv from the command line with an empty vfs cache.

Results: around 8 seconds between the time I press play and the time the file starts playing. I ran a series of 10 tests and the results ranged from 7s to 11s with 8s being the most typical.

rclone2.log (15.2 KB)

[asdffdsa (jojothehumanmonkey)]

ok, that is a good debug log, just not sure what is causing those multiple time delays.

imho, not a rclone issue, not a set of flags to optimize playback.
as this is a small audio file, should be very easy to stream it.

either there is some issue with the media player,
or funky internet connection with that large ping value.
or koofr and pcloud are not optimized for streaming.

can you try using gdrive or some well known providers that works well for streaming.

else we do have rclone mount debug log experts, one will stop by soon.

[ncw (Nick Craig-Wood)]

Here is the first read reading the start of the file

2022/03/06 17:22:39 DEBUG : &{path/file.mp3 (rw)}: Read: len=16384, offset=0
2022/03/06 17:22:39 DEBUG : path/file.mp3(0xc00077c100): _readAt: size=16384, off=0
2022/03/06 17:22:39 DEBUG : path/file.mp3(0xc00077c100): openPending:
2022/03/06 17:22:39 DEBUG : path/file.mp3: vfs cache: checking remote fingerprint "3467402,2020-03-12 18:33:52 +0000 UTC" against cached fingerprint ""
2022/03/06 17:22:39 DEBUG : path/file.mp3: vfs cache: truncate to size=3467402
2022/03/06 17:22:39 DEBUG : path: Added virtual directory entry vAddFile: "file.mp3"
2022/03/06 17:22:39 DEBUG : path/file.mp3(0xc00077c100): >openPending: err=<nil>
2022/03/06 17:22:39 DEBUG : vfs cache: looking for range={Pos:0 Size:16384} in [] - present false
2022/03/06 17:22:39 DEBUG : path/file.mp3: ChunkedReader.RangeSeek from -1 to 0 length -1
2022/03/06 17:22:39 DEBUG : path/file.mp3: ChunkedReader.Read at -1 length 4096 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:39 DEBUG : path/file.mp3: ChunkedReader.openRange at 0 length 134217728
2022/03/06 17:22:40 DEBUG : path/file.mp3: ChunkedReader.Read at 4096 length 8192 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:40 DEBUG : path/file.mp3: ChunkedReader.Read at 12288 length 16384 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:40 DEBUG : path/file.mp3: ChunkedReader.Read at 28672 length 32768 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:40 DEBUG : path/file.mp3(0xc00077c100): >_readAt: n=16384, err=<nil>
2022/03/06 17:22:40 DEBUG : &{path/file.mp3 (rw)}: >Read: read=16384, err=<nil>

That looks OK.

The second read reads a bit more from the start of the file

2022/03/06 17:22:40 DEBUG : &{path/file.mp3 (rw)}: Read: len=32768, offset=16384
2022/03/06 17:22:40 DEBUG : path/file.mp3(0xc00077c100): _readAt: size=32768, off=16384
2022/03/06 17:22:40 DEBUG : vfs cache: looking for range={Pos:16384 Size:32768} in [{Pos:0 Size:28672}] - present false
2022/03/06 17:22:40 DEBUG : path/file.mp3(0xc00077c100): >_readAt: n=32768, err=<nil>
2022/03/06 17:22:40 DEBUG : &{path/file.mp3 (rw)}: >Read: read=32768, err=<nil>

Again all OK here.

The 3rd read, however reads from the end of the file.

2022/03/06 17:22:40 DEBUG : &{path/file.mp3 (rw)}: Read: len=4096, offset=3465216
2022/03/06 17:22:40 DEBUG : path/file.mp3(0xc00077c100): _readAt: size=4096, off=3465216
2022/03/06 17:22:40 DEBUG : vfs cache: looking for range={Pos:3465216 Size:2186} in [{Pos:0 Size:61440}] - present false
2022/03/06 17:22:41 DEBUG : path/file.mp3: ChunkedReader.Read at 126976 length 131072 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:41 DEBUG : path/file.mp3: ChunkedReader.Read at 258048 length 262144 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:42 DEBUG : path/file.mp3: ChunkedReader.Read at 520192 length 524288 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:43 DEBUG : path/file.mp3: ChunkedReader.Read at 1044480 length 1048576 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:45 DEBUG : path/file.mp3: ChunkedReader.Read at 2093056 length 1048576 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:47 DEBUG : path/file.mp3: ChunkedReader.Read at 3141632 length 1048576 chunkOffset 0 chunkSize 134217728
2022/03/06 17:22:48 DEBUG : path/file.mp3(0xc00077c100): >_readAt: n=2186, err=EOF
2022/03/06 17:22:48 DEBUG : &{path/file.mp3 (rw)}: >Read: read=2186, err=<nil>

What is interesting about this is that instead of seeking, rclone downloads the entire file. This is because the size of the file is less than --buffer-size.

It seems to take 8 seconds to download the file though which is suprising for a 3 MiB file.

You have 30Mb/s download which is about 3.7 MiB/s so we should expect that file to download in less than one second.

How long does it take using rclone copy to download that file?

[ioorclone]

@ncw

How long does it take using rclone copy to download that file?

This is the part that has me confused as well. I noted above that an rclone copy takes around 2-3 seconds.

Also to rule out bandwidth as the cause, I repeated the test on a separate internet connection which is around 20MiB/s. I got a similar result: 2-3 seconds for rclone copy, around 8 seconds to play. I also repeated the test with a second remote back end, again same results.

@ncw

The second read reads a bit more from the start of the file

Is this second read continuing where the first read finished or does it start over? It looks like the second read is reading from the parts that were already downloaded in the buffer memory. Is this correct or does it download again?

@ncw

What is interesting about this is that instead of seeking, rclone downloads the entire file. This is because the size of the file is less than --buffer-size.

From Global Flags, the default --buffer-size is "16Mi" does this mean Megabits or MegaBytes?

Why does the third read download the whole file when the first two did not? Does the --buffer-size apply to all reads?

Is it possible to have the 3rd read seek instead of downloading? Perhaps that would be quicker.

Are there any other settings you would recommend I try decrease the time to play?

Can I configure rclone so that it downloads the entire file on the first read then any subsequent reads can access the file from memory or would this actually increase the time before the file can start playing? Are there any other flags or configurations I can focus on?

I am not certain about this but I got the impression that setting vfs cache to writes or minimal might have been slightly faster (maybe 1 or 2 seconds) but then there is a predictable buffer skip around 2-3 seconds after the file starts to play. Does it make sense that writes or minimal might be faster or should full always be more performant?

[ncw (Nick Craig-Wood)]

@ioorclone

@ncw

How long does it take using rclone copy to download that file?

This is the part that has me confused as well. I noted above that an rclone copy takes around 2-3 seconds.

I suspect this might be more down to your provider than rclone. Maybe your provider is looking at the headers and throttling the download....

Try setting --vfs-read-chunk-size 0 which will turn off the chunking then this should make the read in the mount identical to rclone copy.

@ioorclone

@ncw

The second read reads a bit more from the start of the file

Is this second read continuing where the first read finished or does it start over? It looks like the second read is reading from the parts that were already downloaded in the buffer memory. Is this correct or does it download again?

It is probably reading from the memory buffer here. The important thing is that it didn't need to seek.

@ioorclone

@ncw

What is interesting about this is that instead of seeking, rclone downloads the entire file. This is because the size of the file is less than --buffer-size.

From Global Flags, the default --buffer-size is "16Mi" does this mean Megabits or MegaBytes?

Bytes. All units in rclone are Bytes not Bits. So you could try reducing this to 0 or 1M say and see what happens, That will cause the 3rd read to seek instead of downloading. That may not speed things up though as seeks can be very slow as they have to re-open the connection.

@ioorclone

Can I configure rclone so that it downloads the entire file on the first read then any subsequent reads can access the file from memory or would this actually increase the time before the file can start playing? Are there any other flags or configurations I can focus on?

That is pretty much what it is doing at the moment, but it is downloading it very slowly for some reason!

@ioorclone

I am not certain about this but I got the impression that setting vfs cache to writes or minimal might have been slightly faster (maybe 1 or 2 seconds) but then there is a predictable buffer skip around 2-3 seconds after the file starts to play. Does it make sense that writes or minimal might be faster or should full always be more performant?

If you don't use full then rclone will seek the file rather than download it all. Subsequent plays will be equally slow though as it isn't cached.

This could be faster in your scenario but it normally it is about the same speed.