Rclone vs acd_cli speeds

Help and Support
1

Hi,

I have been using acd_cli for a couple of months, I did test rclone earlier but the lake of seeking stopped me, this now seems to be working find in rclone beta, and having the encryption built in rather than using encfs seems like a big plus.

(I am suing rclone v1.34-02-g5f320ccβ)

I am using this solution to host media files to plex, and I am seeing a noticeable speed difference between rclone and acd_cli, particularly at the start of playback.

Setup.
acd_cli mount is using encfs encryption layer.
rclone is using build in crypt.

Times are from hitting the play button to the video starting. All the same episode, h264 encoded, no transcoding happening on the server.

local content
3-4 secconds

acd_cli mount
6-8 seconds

rclone mount
10-12 seconds

I am not seeing heavy link utilisation on my connection (100/40 Mbps) during playback, downloading burst to about 20Mpbs very breifly at the start of play, then drops to sit around 5Mpbs. Also data doesn’t appear to start till after the delays above, so it would appear to be some internal process to the apps causing the delay, not the downloading.

I have also tested using rclone under a encfs mount, in place of acd_cli and I see similar load times, so this doesn’t seem linked to the encryption layer of rclone.

I have tried various setting for --max-read-ahead but that have had no effect.

I have run it with verbose, and it seems there is quite a delay between hitting play and the file being loaded (5 seconds if I am reading the logs right), to me it seems like it does some directory stuff before playing the file, --dir-cache-time could help, but it could be days between file playback, would it be possible to have a long term cache of directories?

Any advice on how else I might speed up rclone in this scenario?

Thanks,
Wob

2

Log http://pastebin.com/nt4ts2VU

3

The streaming experience with a mounted rclone drive is for me the main problem and I’m happy to test with any settings or beta builds that can create a smooth streaming experience.

Currently I’m using this:
rclone mount --max-read-ahead 200M --checkers 40 --read-only --allow-other acd: /folder/folder

I have few problems when using VLC but Plex isn’t too happy. It loads alright but after a few minutes it often doesn’t work anymore.

4

Interesting stats thanks.

How big is the file you are testing? Is it bigger than 10GB? If so can you try a file that is smaller than 10GB

If you’ve got the stomach for looking at lots of logs, then it would be worth while looking at the requests rclone makes to ACD. Add the --dump-headers and -v to your command line. It might be that rclone is making requests it shouldn’t do.

I didn’t see anything obviously wrong here.

Hmmm, that is interesting. I wonder what is going on. Any clues?

5

Can you try the amount mount command?:

rclone mount --max-read-ahead 200M --checkers 40 --read-only --allow-other acd: /folder/folder

It should improve your performance. And can you also test with rclone + encfs? This is my current setup and works well for me. My load time is higher than yours, but rclone was faster than acd_cli for me with the above settings.

6

I’d like to just confirm what Wob76 has said.
I see similar load/seek times betweeen rclone and an acd_cli mount. Rclone about 2x as long as an acd_cli mount. acd_cli w/ encfs, rclone with its crypt. haven’t tested with encfs on an rclone mount yet though.

I’ve used the max-read-ahead and checkers option as well and I think it may have sped it up a bit, but still not as fast as an acd-cli mount. Larger files are more effected I would say. If I try a 1Gb tv show file, it’ll be quicker start/seek times than my 10Gb Movie file.

Thanks for all the work you are doing with Rclone.

7

@ladude626

When you say load time is higher, but rclone is faster with encfs, which do you mean?
rclone/encfs is faster for both start and seek times than acd_cli/encfs or only seek or ?

8
I'm striking my prior reply because I realized this is ENCFS over rclone. IAlthough acdcli performs better under that scenerio, i've abandoned that and now use crypt with rclone and speeds is pretty great. Not sure about with Plex though as I use XBMC.
9

Hi ncw,

First off, thanks for all your hard work. The files I am using are fairly small, as I have uploaded them using both encfs and crypt encryption. They range from 140Mb to 350Mb. I’ll try some extra logging shortly.

Re my previous log, It was not so much the “errors” I could find, but more the delay between the start (when I hit play) and when it starts pulling the file, some 5 seconds, can this be sped up, seems maybe a local directory cache could speed up the start of the process?

The only real difference between your command and mine is the read-only. I have tried varies values for --max-read-ahead, with no noticeable change.

Re the questions of encfs over rclone, see my original post.

Thanks All,
Wob

10

@ninthwalker I saw improvement to start time, time to resume a video, and time to seek within a video. I believe http seek is fixed for acd mounts, but I’m not sure for crypt mounts. Maybe @ncw can answer this question.

11

I think a long term directory cache that is updated at X intervals & at every remount would speed things up considerably.

Have you considered doing something like that @ncw ?

Edit: How does --dir-cache-time duration work? I can tell that my folder content is pulled on request & doesn’t just open immediately. So is the entire file structure cached on a mount? Only requested folders/files are cached? Could I initiate a cache of the entire file structure & cache it for an X amount of time & then have it refreshed?

12

If you look at the logs, the dir listings isn’t what is taking time. The log starts at 10:07:33 and the first file open is also at 10:07:33. To me it looks like the pre-work that plex is doing to play the video. I assume playback was pressed at started around 10:07:33. It then opened and closed that file 3 times. You can then see it download in increasing sizes and forward to the end of the file and skip back to the beginning beginning to play… I’m assuming its doing some streaming test to see how fast it can stream to your client. Then it starts. That process take 9 seconds.

The question is still why that portion is slower on rclone than acdcli.

2016/11/09 10:07:33 Dir.Lookup
2016/11/09 10:07:33 Dir.Lookup OK
2016/11/09 10:07:33 File.Attr
2016/11/09 10:07:33 Dir.Lookup
2016/11/09 10:07:33 Dir.Lookup OK
2016/11/09 10:07:33 File.Attr
2016/11/09 10:07:33 Dir.Lookup
2016/11/09 10:07:33 Dir.Lookup OK
…
> directory look ups.
…
2016/11/09 10:07:33 : File.Open
2016/11/09 10:07:35 : ReadFileHandle.Release OK
2016/11/09 10:07:35 : File.Open
2016/11/09 10:07:37 : ReadFileHandle.Release OK
2016/11/09 10:07:37 : File.Open
2016/11/09 10:07:38 : ReadFileHandle.Read size 16384 offset 0
2016/11/09 10:07:38 : ReadFileHandle.Read size 32768 offset 16384
2016/11/09 10:07:38 : ReadFileHandle.Read size 65536 offset 49152
2016/11/09 10:07:39 : ReadFileHandle.Read size 131072 offset 114688
2016/11/09 10:07:39 : ReadFileHandle.Read size 131072 offset 245760
2016/11/09 10:07:39 : File.Attr
2016/11/09 10:07:39 : ReadFileHandle.Read size 4096 offset 314191872
2016/11/09 10:07:39 : ReadFileHandle.seek from 376832 to 314191872
2016/11/09 10:07:40 : File.Attr
2016/11/09 10:07:40 : ReadFileHandle.Read size 131072 offset 376832
2016/11/09 10:07:40 : ReadFileHandle.seek from 314195200 to 376832
2016/11/09 10:07:41 : ReadFileHandle.Read size 131072 offset 507904
2016/11/09 10:07:42 : File.Attr
2016/11/09 10:07:42 : ReadFileHandle.Read size 131072 offset 638976
> <<<<-- This looks like where playback actually starts.
2016/11/09 10:07:42 : ReadFileHandle.Read size 131072 offset 638976
…
… continued 20 times at the next offset of size 131072
…
2016/11/09 10:07:48 : ReadFileHandle.Read size 131072 offset 3260416
2016/11/09 10:07:48 : ReadFileHandle.Flush
2016/11/09 10:07:48 : ReadFileHandle.Flush OK
2016/11/09 10:07:48 : ReadFileHandle.Flush
2016/11/09 10:07:48 : ReadFileHandle.Flush OK
2016/11/09 10:07:48 : ReadFileHandle.Flush
2016/11/09 10:07:48 : ReadFileHandle.Flush OK
2016/11/09 10:07:48 : ReadFileHandle.Release closing
2016/11/09 10:07:48 : ReadFileHandle.Release OK
2016/11/09 10:08:01 : Dir.Attr

13

The cache simply caches what has been been read for that period of time and then drops it. There is a issue out there to have a way to make this number really large and then allow a user to ‘purge’ that cache on demand via the command line. That would help if you understand the activity on your mount.

It would be an interesting option to combine that with a ‘scan’ of the entire file system to recache it rather than just drop the cache at the end of the interval. I believe that is what you mentioned. I actually like the idea. It would be rather simple to recurse the tree on whatever interval is selected to ‘recache’ it as an option.

That being said, from the logs above, I do not see the directory caching as the issue based on the timings.

14

rclone has a directory cache of 5 minutes by default. Is it possible it is expiring? You could try raising it with --dir-cache-time 1h say

15

When I open my mounted directory I have to wait a little for each folder I try to open. When I open all folders it keeps that structure in memory for 5min by default. So what happens after 5min? @calisro mentioned that the cache is dumped. Will it reload it automatically or only when there is an attempt to access that folder?

16

I’ll have a play with --dir-cache-time and see if that helps, but just to be clear, if I set that it would take rclone an hour to see a new file in that folder?

As I am only likely to access files from ACD once a day or even once a week it would still need to cache of first access? Could I increase it to say 30min and then run a find over the folder say every 35 min, would this then cause rclone to keep the folder structure in memory semi-permanently?

if calisro is correct and the directory cache is totally unrelated, is there anything more I can do to help diagnose? I had thought the cache could be the issue as I know acd_cli keeps a local db for directory listing, it does have the short coming of requiring a command to update said cache to see new files. Would a log from acd_cli be of any benefit?

Just a note, I did some testing yesterday, and the speed difference is not hugely noticeable if you are not looking for it. I had 2 remote clients streaming a 5Gb movie at the same time, and didn’t see any bandwidth issues.

Thanks,
Wob

17

If you run a find, it will recache. So yes you could do that. I doubt it would make a noticable difference though in your use-case at least.

From what I see in the log, a local cache would help. Those 3 file reads would be faster.

18

After 5 minutes the data is dropped from memory.

Only when there is an attempt to read that directory

19

Yes that is correct.

Yes something like that would work. You’d probably want to set the cache time to 30m say, but run the find every 15mins.

At the moment the directory cache isn’t very clever. When the directory is read rclone stores the time. When it is accessed again, if the current time is more than the cache time later then it re-reads the directory from the remote.

rclone could do this a lot more cleverly if this turns out to be the problem - keeping used directories in memory and incrementally updating them instead of ditching them completely.

So please experiment and we’ll find out whether it is worth improving the directory cache.

acd_cli keeps its directory cache on disk and it does clever updates of it - rclone could eventually grow a system like that.

I’d like to rule in or out the directory caching first!

20

Right now I feel like I need to combine a couple of things to make mounting usable with plex:
(1) Run a mount command with modifiers for cache time & max read ahead
-> rclone mount --max-read-ahead 200M --dir-cache-time 30m --checkers 40 --read-only --allow-other acd: /folder/folder

(2) Run a find every 15mins as recommended by @ncw
Question: what’s the easiest way to reload the entire folder/file structure on a mac?

(3) Run a umount once in awhile to fix the problem when the mount becomes inaccessibility (& remount right after)
-> run umount -f /folder/folder

I hope the list won’t be getting too much longer :slight_smile: