My VFS SweetSpot - Updated 30-Aug-2018


#301

Thanks, I’ve just created my own API with that little tutorial. Hopefully I can see some API hits tomorrow. Will update later.

@BinsonBuzz: I’ve moved from a 200/20 to a 500/500 so I expected at least some more speed, so it’s weird to me. I will try your advise for the changed buffer!

For the rclone rc command, do you run it as a standalone snippet right after the mount or is it integrated in a longer string?

Currently I’m not testing with Plex but with Emby, since previously I had better results with that. My library scan just finished on Emby so I will test some more tomorrow. Then I will also compare it to Plex with your throttle buffer setting.


#302

As a standalone snippet right after the mount - that’s the full string


#303

I think I didn’t do it right yesterday with my own API, didn’t get any hits. So I created the API and deleted my rclone mount and tried to reconfigured. So I go through the rclone wizard, with client ID filled it from my dashboard. Then I get the 0auth screen and paste in the code. Then I get the following error:

2018/11/06 09:59:38 Failed to configure token: failed to get token: oauth2: cannot fetch token: 401 Unauthorized
Response: {
“error”: “invalid_client”,
“error_description”: “Unauthorized”
}

So could it be that my 0auth hasn’t been processed by Google yet, or am I doing something wrong? I only got asked for the Client ID during rclone config, not for the client password.

EDIT: just went through the Rclone config for the 5th time and this time it did ask for my client secret from the start. So now I’ve got this configured. Hopefully I will get api hits now.

@BinsonBuzz

I’ve put this code behind my mount but it gives me the following error:
Failed to rc: connection failed: Post http://localhost:5572/vfs/refresh: dial tcp 127.0.0.1:5572: connect: connection refused

My code looks like this now:

rclone mount --allow-other --dir-cache-time 72h --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off --buffer-size 100M --log-level INFO gdrive_crypt: /mnt/user/mount_rclone/Gdrive --stats 1m &

rclone rc --timeout=1h vfs/refresh recursive=true


#304

Sorry for dirtying up the thread. I’ve got the API working and the performance now is amazing! Startup times are within 5 seconds for my short tests with Emby Will also try Plex, see how it goes there. This feels like my previous performance again. Really amazing how big of a difference this makes!

Now if I could get that RC working then I’m pretty much set again I hope :).


#305

hmm just checked my logs and I have the same - it was working before. Will investigate tonight.

Glad it’s working - basically it’s about the same as spinning up a local disk :slight_smile:

I’ve got 80% of my media on gdrive now and I only really keep some of my 4K stuff local just to remove the bandwidth requirements, not because rclone can’t cope


#306

What did you use in the end for your mount BTW? drive-chunk-size 32M or 128M?


#307

Currently testing with 128m. But will try 32m later today. What made you switch back from 128 to 32?


#308

Sorry, I meant vfs-read-chunk-size 64M Vs your vfs-read-chunk-size 128M

I went with 64M as I was trying to get faster start times, but I might test again with 128M as I got upgraded last month from 200/200 to 300/300, so it might be a bit faster.


#309

Happy to hear it’s working well for you. Using your own API key will definitely do better.


#310

I saw you somewhere post your Plex settings to stop analyzing. I browsed through your posts, but can’t find it anymore. Would you mind sharing it here again? I’m noticing Emby performs quite a lot better than Plex, which shouldn’t happen.


#311

Well, it does have to analyze all the files in both Plex and Emby.

I keep all the thumbnails/previews/etc off:

Everything else really doesn’t matter too much with the exception of deep analyze. The box is extensive media analysis.

Emby does a ffprobe when it adds a file to the library for every item to get the information which is similiar to what Plex does as well.

I use a script to check the analyzed files in Plex:

felix@gemini:~$
felix@gemini:~$ plex-library-stats.sh
06.11.2018 09:00:41 PLEX LIBRARY STATS
Media items in Libraries
Library = Movies
  Items = 1977

Library = TV Shows
  Items = 20295

Library = xMMA
  Items = 53

Library = zExercise
  Items = 279

Time to watch
Library = Movies
Minutes = 211705
  Hours = 3528
   Days = 147

Library = TV Shows
Minutes = 816201
  Hours = 13603
   Days = 566

Library = xMMA
Minutes = 9240
  Hours = 154
   Days = 6

Library = zExercise
Minutes = 11278
  Hours = 187
   Days = 7

22731 files in library
0 files missing analyzation info
0 media_parts marked as deleted
0 metadata_items marked as deleted
0 directories marked as deleted
22606 files missing deep analyzation info.

Once that files missing analyze is 0, Plex scans are just about instant (10-20 seconds).


#312

Thanks! That made me change some settings (not a lot). So I will test a bit with the read chunk size to see if that speeds up playing time. But I think it doesn’t make much a problem. At this point the mediaplayer device is probably the only limitation.


#313

If I am not misreading it, you probably need to add the --rc argument to the rclone mount for the rc commands to be available.


#314

yep - fixed it:

rclone mount --rc --rc-addr=172.30.12.2:5572 --allow-other --buffer-size 1G --dir-cache-time 72h --drive-chunk-size 32M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_media_vfs: /mnt/user/mount_rclone/google_vfs


#315

How did you come up with rc-addr=172.30.12.2:5572 is that the error you got back? I got a different IP, that’s why Im asking.


#316

The IP address he is used is more related to his environment.

If you plan to run the rc commands from the same box, you can just use --rc and it’ll bind to the local loopback, which means only that box can access it.

I just use the --rc and you can run it like:

felix@gemini:~$ rclone rc vfs/refresh recursive=true

#317

What am I missing here then, this my error:

2018/11/06 15:46:56 NOTICE: Serving remote control on http://127.0.0.1:5572/
2018/11/06 15:46:56 ERROR : rc: “vfs/refresh”: error: couldn’t find method “vfs/refresh”
2018/11/06 15:46:56 Failed to rc: Failed to read rc response: 405 Method Not Allowed: {
“error”: “couldn’t find method “vfs/refresh””,
“input”: {
“recursive”: “true”
}
}

My rclone mount:
rclone mount --rc --allow-other --buffer-size 1G --dir-cache-time 72h --drive-chunk-size 32M --fast-list --log-level INFO --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off gdrive_crypt: /mnt/user/mount_rclone/Gdrive --stats 1m &

And the seperate code after the mount:
rclone rc vfs/refresh recursive=true


#318

What does rclone version show?


#319

rclone v1.43-150-g249a523d-beta

  • os/arch: linux/amd64
  • go version: go1.11.1

#320

Update to 1.44 which is the latest version.