Rclone and Nextcloud buffer issues

rejectedpeice · June 28, 2021, 5:15am

What is the problem you are having with rclone?

I am trying to use rclone mount with nextcloud. When I play a video file, the issue is that the file is getting downloaded to the system, even after the playback is stopped. This results in sluggish performance when using nextcloud

What is your rclone version (output from `rclone version`)

v1.50.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20.04.2 LTS x86_64

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

 rclone mount rejected:Folder /mnt/test  --allow-other --uid 1000 --umask 000 --cache-chunk-size=32M --drive-chunk-size=32M --buffer-size=256M --no-gzip-encoding --allow-non-empty --no-checksum --vfs-read-chunk-size=64M --vfs-cache-mode writes  -vv

The rclone config contents with secrets removed.

type = drive
scope = drive
acknowledge_abuse = true

A log from the command with the `-vv` flag

2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: ChunkedReader.Read at 2715648 length 1048576 chunkOffset 622592 chunkSize 67108864
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=1794048
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=1925120
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2056192
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2187264
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2318336
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2449408
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2580480
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2711552
2021/06/28 07:12:23 DEBUG : Folder/AnotherVideo.mp4: ChunkedReader.Read at 2093056 length 1048576 chunkOffset 0 chunkSize 67108864
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1163264
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1294336
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1425408
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1556480
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1687552
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1818624
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=1949696
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2080768
2021/06/28 07:12:23 DEBUG : Folder/: Lookup: name="04 - Planning.mp4"
2021/06/28 07:12:23 DEBUG : Folder/: >Lookup: node=Folder/04 - Planning.mp4, err=<nil>
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: Attr:
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: >Attr: a=valid=1s ino=0 size=15585258 mode=-rw-rw-rw-, err=<nil>
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: Open: flags=OpenReadOnly
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: Open: flags=O_RDONLY
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: >Open: fd=Folder/04 - Planning.mp4 (r), err=<nil>
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: >Open: fh=&{Folder/04 - Planning.mp4 (r)}, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/04 - Planning.mp4 (r)}: Read: len=16384, offset=0
2021/06/28 07:12:23 DEBUG : Folder/04 - Planning.mp4: ChunkedReader.openRange at 0 length 67108864
2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: ChunkedReader.Read at 68153344 length 1048576 chunkOffset 67108864 chunkSize 134217728
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67223552
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: Attr:
2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: >Attr: a=valid=1s ino=0 size=201736357 mode=-rw-rw-rw-, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67354624
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67485696
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67616768
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67747840
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=67878912
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68009984
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68141056
2021/06/28 07:12:23 DEBUG : Folder/AnotherVideo.mp4: ChunkedReader.Read at 68153344 length 1048576 chunkOffset 67108864 chunkSize 134217728
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67223552
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : Folder/AnotherVideo.mp4: Attr:
2021/06/28 07:12:23 DEBUG : Folder/AnotherVideo.mp4: >Attr: a=valid=1s ino=0 size=135228821 mode=-rw-rw-rw-, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67354624
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67485696
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67616768
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67747840
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=67878912
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68009984
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68141056
2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: ChunkedReader.Read at 3764224 length 1048576 chunkOffset 622592 chunkSize 67108864
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2842624
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=2973696
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3104768
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3235840
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3366912
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3497984
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3629056
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=3760128
2021/06/28 07:12:23 DEBUG : Folder/AnotherVideo.mp4: ChunkedReader.Read at 3141632 length 1048576 chunkOffset 0 chunkSize 67108864
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2211840
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2342912
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2473984
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2605056
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2736128
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2867200
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=2998272
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=3129344
2021/06/28 07:12:23 DEBUG : Folder/Video1.mp4: ChunkedReader.Read at 69201920 length 1048576 chunkOffset 67108864 chunkSize 134217728
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68272128
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68403200
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68534272
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68665344
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68796416
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=68927488
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=69058560
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:23 DEBUG : &{Folder/Video1.mp4 (r)}: Read: len=131072, offset=69189632
2021/06/28 07:12:24 DEBUG : Folder/AnotherVideo.mp4: ChunkedReader.Read at 69201920 length 1048576 chunkOffset 67108864 chunkSize 134217728
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68272128
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68403200
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68534272
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68665344
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68796416
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=68927488
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=69058560
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: >Read: read=131072, err=<nil>
2021/06/28 07:12:24 DEBUG : &{Folder/AnotherVideo.mp4 (r)}: Read: len=131072, offset=69189632

Animosity022 · June 28, 2021, 10:16am

Rclone doesn't download anything on its own as it only does whatever the client/application asks for.

You have an old version so you'd want to update.

You probably want to take a look at:

https://rclone.org/commands/rclone_mount/#vfs-cache-mode-full

rejectedpeice · June 29, 2021, 8:44am

I just tried VFS cache mode full, it kinda made the experience, but not perfect. I will hit nextcloud community for this same issue, and see if I can get a better solution.

Also, all I am trying to achieve is video streaming from my home server (videos are in drive). Sometimes, videos in drive never gets processed, hence I am looking to set my own setup. If you have any better options, you can reply here.
I haven't tried plex yet, mostly because it doesn't allow sharing of videos with others (like enter a link, and you are good to go).

system · August 29, 2021, 4:45am

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.