Good day
I'm running an application on AWS over several Windows 2016 servers with the support of rclone and winfsp
I'm using start rclone mount media:mediastorage/ C:\MediaStorage --vfs-cache-mode full --allow-other to mount my S3 bucket
One server uploads the media files via a web application, using regular file copy
When the other tries to read it, it hangs for several minutes before it is finally able to read
The rclone mount is performed on system startup, well before the said file would have been written to S3
I found out that if i were to stop rclone and restart it, I am able to access the files instantly. This however, is not a viable option as other operations may be going on at the same time so i can't just unmount and re-mount arbitrarily
Is there something i'm doing wrong?
Thanks
Adewale_Ajiboye:
--vfs-cache-mode full
This means that is must download the whole file before it's accessible.
You can read up here:
https://rclone.org/commands/rclone_mount/#file-caching
I'm guessing that isn't what you want.
Hi, and thanks for your reply
The file isn't very large. Only about 2MB, so its not a problem if it has to download the entire file. I'm trying to copy the entire file to a local directory anyway
Can you replicate the issue and have the mount with -vv and share the debug log? We can see what it is doing.
So, this is part of the log:
2020/02/22 16:06:49 DEBUG : /UnprocessedStreams/305/1582382963.461: >Opendir: errc=0, fh=0x3
basically, it stays at that last line for a really long time
After that, i got:
2020/02/22 16:18:42 INFO : Cleaned the cache: objects 6 (was 6), total size 2.913M (was 2.913M)
At this point, the file read has still not gone through
ncw
February 22, 2020, 3:45pm
7
Adewale_Ajiboye:
basically, it stays at that last line for a really long time
Hmm, strange.
Can you use rclone copy to copy the file OK or does it have the same hang?
Could there be an antivirus interfering somehow?
Hi ncw
No antivirus on the servers
And windows defender scanning is disabled.
What is strange is if i stop rclone and re-mount, i get instant access
Unfortunately, as a result of the deadline, i didn't have time to do too much testing with rclone copy
i went ahead and removed --vfs-cache-mode and modified all my applications to ensure that i don't need to seek while writing
After doing that, everything works fine
Thanks for all your help ncw and Animosity022
ncw
February 24, 2020, 10:53am
9
Adewale_Ajiboye:
i went ahead and removed --vfs-cache-mode and modified all my applications to ensure that i don't need to seek while writing
After doing that, everything works fine
Ah! That was probably the cause then. You should find that --vfs-cache-mode writes works OK too.
system
June 4, 2020, 11:35am
11
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.