Buffering on Kodi high bitrate 4k from my mount

Help and Support
1

What is the problem you are having with rclone?

Having buffering when I am playing on Kodi high bitrate 4k from my mount. No transcoding.

Run the command 'rclone version' and share the full output of the command.

rclone --version
rclone v1.61.1

  • os/version: debian 9.13
  • os/kernel: 4.9.268-kc375 (armv7l)
  • os/type: linux
  • os/arch: arm
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

cat rclone-mount.service
[Unit]
Description=RClone Service
After=network-online.target
Wants=network-online.target


[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/john/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount gdrive9: /home/john/mnt/GD \
--allow-other \
--umask 002 \
--vfs-cache-mode full \
--cache-dir=/home/john/all \
--no-checksum \
--no-modtime \
--no-seek \
--read-only \
--vfs-read-ahead 1G \
--vfs-read-chunk-size=256M \
--buffer-size=32M \
--log-level DEBUG \
--dir-cache-time=24h \
--vfs-cache-max-age 6h \
--vfs-cache-max-size 300G \
--log-file /home/john/rclone.log

ExecStop=/bin/fusermount -u -z /home/john/mnt/GD
Restart=on-failure
User=root
Group=root


[Install]
WantedBy=default.target

The rclone config contents with secrets removed.

cat rclone.conf
[gdrive9]
type = drive
client_id = XXX
client_secret = XXX
token = XXX

A log from the command with the -vv flag

https://pastebin.com/S6huh7ab

Hi fellas, I am having buffering when I am playing on Kodi high bitrate 4k from my mount. No transcoding.
In this case, I am trying to play The Lord of the Rings: The Fellowship of the Ring (Uhd Remux 4k around 140GB)

I recently got a low end dedicated server in OVH Datacenter 250mbps.
Its write speed is around 100 Mb/s so it should be okay.

I got the same file in the server hard drive and there is not issue, playback is perfect, but with the mount it is buffering every minute.

Then, when I check the iotop I can see 99.9% I/O is being used by "vsftpd". So, I know that the issue and its because the system is a low end, and it cannot manage that load. I get that, but I would like to know if:

Is there any way I could improve and remove the buffering? Any tuning?

I have been trying and changing all the flags and at the moment I did not get any improving.
I dont know if I am missing something. I dont know how to read the logs. This is why I am asking for help

  • Log file was around 6 mb, noone of the webs were allowing me to upload it, so I eventually cut it a bit ( if you need the full log I can send it to you, I dont know how but we can do it if its necesary)

Many thanks

Relevant Info:

  • My location is in Spain and the Server is in France.
    Home connection: Fibre 500/500 mbps, conected from router to the android box by ethernet Cat 6.
    I get in the android box 480mbps according with speedtest.net

  • Cloud Server: Google Drive (own-client-id set up)
    Program used: Kodi player (Kodi settings changed to avoid any buffering in advancedsettings.xml)
    My server is set in Kodi as FTP, so I access to the mounted folder, find the file and start the playback

  • I use a android box tv with Coreelec installed and it is working perfectly with everything, no buffering, only when I load high bitrate 4k movies

When I use "move or copy" on rclone (not from the mount, just the command) I am getting around 150 mbps (18.75 MiB/s) uploading and downloading from Server to Gdrive (checked from vnstat -l)

  • Server Bench:
Processor:    ARMv7 Processor rev 1 (v7l)
CPU cores:    2
Frequency:    MHz
RAM:          2.0G
dd: sequential write speed
    1st run:    112.53 MiB/s
    2nd run:    118.26 MiB/s
    3rd run:    118.26 MiB/s
    average:    116.35 MiB/s
IPv4 speedtests    
    Cachefly CDN:         98.55 MiB/s
    Leaseweb (NL):        47.69 MiB/s
    Online.net (FR):      71.88 MiB/s
2

hi,
i would start testing with a very simple command.
to remove these flags.
--no-seek, not sure i have ever seen that flag used.

--no-checksum \
--no-modtime \
--vfs-read-ahead 1G \
--vfs-read-chunk-size=256M \
--buffer-size=32M \

or use this as a template
https://github.com/animosity22/homescripts/blob/master/systemd/rclone-drive.service

1 Like
3

Thanks for your response.
As I said , I ve been tried with many flags and also with animosty22 template, changing them, increasing, reducing .... and always I got the same issue, buffering.
This is why I came to ask as I dont understand the log very well
Many thanks

4

i did not see anything of use in that short log snippet.
looks for error, pacer, retry, retries, i/o
or post the full log

5

If the system is disk IO bound, you can't tune around that.

Reduce disk IO.

6

Thanks for the responses
I agree, this is what im trying to do.
These 99.9% i/o is for the rclone mount, trying to get the file from Gdrive and sending to my Kodi player

Any help for tuning this mount?
How can I post 7 mb log? All webs are making some error for excesing the max size
Where can I post that huge log?

Many thanks

7

You'd have to share more what you are looking at as I'm not sure what you mean by I/O 99%.

You can use your Google drive to share a link to it.

8

This is the full log https://drive.google.com/file/d/1caIqXBfeMiXsbQ0o-lRHrtK17jc5zprT/view?usp=share_link

I explained in the first post that during the playback I got the buffering and 99.9 I/O on iotop from the ftp client (vsftpd) when rclone is trying to get the file from Gdrive and sending to my Kodi player

when I check the iotop I can see 99.9% I/O is being used by "vsftpd". So, I know that the issue and its because the system is a low end, and it cannot manage that load

Many thanks for the support

9

I think we are going in a bit of a circle.

You said the mount was IO but you said now that it’s VSFTP.

If the system is IO bound, any thing using disk will be slow and sluggish. You gotta fix the IO problem before you can get the mount working.

10

Vsftp is the ftp app I use in kodi to see and play the mount. The system is ok until I play any 4k movie, it´s when the IO get high. Something is wrong with the GD mount that the system cant get the playback smooth. But the system is not bound, just in this scenario.
Thanks

11

If the system is IO bound, you’ll have issue using the mount.

12

Are there anything I can use to reduce the load on the rclone mount?
Is anything wrong in the log?
There are many low end systems that can handle a high load mount. This is what I m looking for.

Many thanks

13

You said the load was vsftpd.

Please share a screenshot or something as what you are saying makes no sense.

14

Normal state

High Load when I start the playback

I hope now it makes sense

Many thanks for looking into

15

RIght so what does VSFTPD have to do with your rclone mount?

image
image770×178 50.6 KB

That shows that VSFTPD is your bottleneck so that is what needs to be fixed.

16

Any help with that?
Thanks

17

Get a better disk, better server, etc.

The rclone mount has really nothing to do with it as you are disk bound.

18

The point is that I got the same 4k movie in the HDD and when I play that movie vsftpd I/O is 5%
I get the high IO only when I play it from the Rclone mount. The disk is bound only in this case
Thanks

19

Sorry as I have nothing else to add/offer as the disk IO is your root cause.

Nothing in rclone to tune to get around slow disk.

You hit a point that you can't sustain on slow disk.

20

Thanks for the support but I dont think that this is a slow disk

dd: sequential write speed
    1st run:    112.53 MiB/s
    2nd run:    118.26 MiB/s
    3rd run:    118.26 MiB/s
    average:    116.35 MiB/s