Mount over samba with a lot files doesn't works play files

dir cache doesn't solve the problem.

your configuration is a little bit tricky.. my config is more simple and doesn't works...

use ubuntu and mount with rclone a drive

then go into win11 for example, then open google drive mounted into ubuntu, via samba.

then open a folder if you have in your google drive, with a lot of files, 11.000 or more and you can get a test like me.

please post the exact command.

86 rclone serve dlna ROB_MASTER: --addr :8080 --vfs-cache-mode=full --dir-cache-time=100h -vv
87 rclone serve webdav ROB_MASTER: --addr :8080 --vfs-cache-mode=full --dir-cache-time=100h -vv
88 rclone serve http ROB_MASTER: --addr :8080 --vfs-cache-mode=full --dir-cache-time=100h -vv
89 rclone serve http ROB_MASTER: --addr :8080 --vfs-cache-mode=full --dir-cache-time=100h -vv

from history

a video of this issue

when i open my movie folder that contain 11.000 files, and i open an mkv, always, always, this story, scan all movies without any sense, and doesn't open a file.

rclone mount ROB_MASTER:/ /mnt/ROB_MASTER/ --vfs-cache-mode=full --allow-other -vv

Google Drive mounted in ubuntu and shared over samba with windows

but same issue with kodi in firestick.

( in kodi i get the folder over samba, shared from ubuntu server )

i have tried all protocols, dlna, webdavb, http, same issue )rclonelog.log (379.6 KB)

  • what was the exact command used for for that video?

  • that rclonelog.log you posted, does match the video, is that correct?

  • from the video, i agree that, at approx 1:00, once you double-click on the video to start playback, rclone is doing a lot of Lookup.

  • at this point, the dir cache should be up to date so rclone should not need to access gdrive, so i assume.

  • so what is requesting the multiple Lookup?
    on the one hand: is that vlc or windows explorer, asking rclone for the files in that current working folder?
    on the second hand: is rclone itself doing the Lookup for some reason?
    on the gripping hand, let's ask @Animosity022 for a deep dive on what is going on!

  • and how can we tell if rclone is asking gdrive for details about a file or if rclone is getting the details from the dir-cache?

I have tried
rclone mount ROB_MASTER:/ /mnt/ROB_MASTER/ --vfs-cache-mode=full --allow-other --log-file=mylogfile.txt --no-checksum --no-modtime --read-only -vv

no success.

  • what was the exact command used for for that video? Yes
  • that rclonelog.log you posted, does match the video, is that correct? yes

This issue is also when you use kodi, not also windows. Every client you use, have this issue

  • the log is from 19:50 to 19:51
  • in the video
    --- at 1:00, the time is 19:55
    --- nothing about chunked reading during media playback

mylogfile.part001.log (4 MB)
mylogfile.part002.log (4 MB)
mylogfile.part003.log (4 MB)
mylogfile.part004.log (4 MB)
mylogfile.part005.log (4 MB)
mylogfile.part006.log (4 MB)
mylogfile.part007.log (1.8 MB)

please rename .log to .rar

Log is 1gb
in the video i didn't use -vv so you can lookup when open an mkv.

i make another log and here i open #Alive (2020).mkv into movies folder that contain 12.016 files.

In this test the first time that i tried to open this movie, no success. So i retry one more time to open it.

When is open i seek to 16mins

Then close it

do you have the log that matches that video.

The log I looked at had nothing being played in it.

I can see in the video explorer and stuff going on so not sure what can be doing as well.

Best to find a very simple test case with a player and use command prompt or something as Windows just does odd things at times and you can't tell what's going on.

For me, I rather make things as simple as can be and invest so I don't have to troubleshoot or work around many layers as it saves a lot of headaches in the end.

sure, i agree.

for a Lookup, is there a way to know if rclone is requesting the dir/file details from the remote or getting that from local dir cache?

I don't think so.

Those calls are all in the backend and I think hidden somewhat as that's all from dir-cache-time

2021/12/06 19:50:52 DEBUG : /: >ReadDirAll: item=16, err=<nil>
2021/12/06 19:50:52 DEBUG : /: Lookup: name="ANIME"
2021/12/06 19:50:52 DEBUG : /: >Lookup: node=ANIME/, err=<nil>

So like that is just getting a directory listing.

The 'easy' way to tell is if it's fast or slow.

In Linux, you can -HUP the process to clear out the dir cache and just measure the timing on them:

felix@gemini:/DB$ time ls | wc -l
92
real	0m0.957s
user	0m0.003s
sys	0m0.002s
felix@gemini:/DB
$ time ls | wc -l
92
real	0m0.004s

So that's a small listing with 92 files and took almost a second to hit the remote but the 2nd one was cached and took a "little" less.

1 Like

thanks,

ok. that is what i thought too.

that is what i rely on as well.

i will try that. i wonder if there is a window equivalent.

Install Linux?

image

amazing, that is exactly what i envisioned what you would look like :wink:

i use WSL2g and use it more and more these days.
in fact, i used it for https://forum.rclone.org/t/getting-bash-command-not-found/27883/6?

https://vimeo.com/654043355

mylogfile.log.part006.log (4 MB)
mylogfile.log.part005.log (4 MB)
mylogfile.log.part004.log (4 MB)
mylogfile.log.part003.log (4 MB)
mylogfile.log.part002.log (4 MB)
mylogfile.log.part001.log (4 MB)
mylogfile.log.part008.log (2.0 MB)
mylogfile.log.part007.log (4 MB)
Please rename from .log to .rar

As you can see from this video, i make a mount with this command:
rclone mount ROB_MASTER:/ /mnt/ROB_MASTER/ --vfs-cache-mode=full --allow-other --log-file=mylogfile.txt --no-checksum --no-modtime --read-only -vv

The first time, so you can't see any shell output because write into the txt file.

As you can see anyway there are a lot of lookup for every files into Movies folder.

And any file can't be played.

But when lookup finished, you can.

But if you try to open another file, here we go again, and lookup again....
_DOWNLOAD folder, and as you can see, works without issue and without any lookup.

Is fast.

This issue is also with kodi, so isn't related with windows.

And anyway every time that i tried to play a movie, lookup denied that, because want to scan all files before.

I'm very despondent, but i guess that there is a solution for that.

I can't believe that isn't possible

That goes to nothing:

image

I'm not sure what that is, can you share a single log file? Use pastebin/make a google drive link/anything.

You have a strange command going on. If you still using google drive, make life easy.

rclone mount ROB_MASTER:/ /mnt/ROB_MASTER/ --vfs-cache-mode=full --allow-other --log-file=mylogfile.txt --read-only -vvv --dir-cache-time 1000h

I have tried also
https://rclone.org/docs/#max-backlog-n

That say file over 10.000, but doesn't change nothing, always this lookup :frowning:

 rclone mount ROB_MASTER:/ /mnt/ROB_MASTER/ --vfs-cache-mode=full --allow-other --no-checksum --no-modtime  --read-only --max-backlog=20000 -vv

video on vimero now works

log is very large, pastebin is too limited.
are 500mb of txt, so is impossible on pastebin
mylogfile.part003.log (4 MB)
mylogfile.part002.log (4 MB)
mylogfile.part001.log (4 MB)
mylogfile.part004.log (2.0 MB)
rename from .log to .rar

In this log i have tried to open 3 Days to Kill (2014).mkv
2 times, and the 2nd, after a lot, was opened.

That's not related to a mount command as it's for copy/move/sync as it says that in the docs.

Use a Google Drive link or some file sharing as it's really painful to combine files and unrar things as I'm not doing that again.

You seem to be using Windows Explorer, which is trying to enumerate out all the files there and doing it's thing.

Then you start to get errors on Samba:

2021/12/07 13:05:15 DEBUG : Movies/3 Days to Kill (2014).mkv: vfs cache: stopping download thread as it timed out
2021/12/07 13:05:16 DEBUG : Movies/3 Days to Kill (2014).mkv: vfs cache: stopping download thread as it timed out

and the file gets opened a closed a bunch.

2021/12/07 13:05:09 DEBUG : Movies/3 Days to Kill (2014).mkv: Open: flags=OpenReadOnly+OpenNonblock+0x20000
2021/12/07 13:05:09 DEBUG : Movies/3 Days to Kill (2014).mkv: Open: flags=OpenReadOnly+OpenNonblock+0x20000
2021/12/07 13:05:09 DEBUG : Movies/3 Days to Kill (2014).mkv: Open: flags=OpenReadOnly+OpenNonblock+0x20000
2021/12/07 13:06:54 DEBUG : Movies/3 Days to Kill (2014).mkv: Open: flags=OpenReadOnly+OpenNonblock+0x20000
2021/12/07 13:06:54 DEBUG : Movies/3 Days to Kill (2014).mkv: Open: flags=OpenReadOnly+OpenNonblock+0x20000

So we're back to Samba/Windows Explorer being a problem which is beyond anything I can help solve as the rclone piece is working just fine.