htusern84
(Ben Dover)
February 18, 2022, 1:35am
1
What is the problem you are having with rclone?
--vfs-cache-mode writes
I'm trying to play some video files and Rclone is trying to completely download it (in the cache folder) before letting it play.
I'm currently on Linux Mint 20.3, I've never had this issue on windows and the option worked normally (caching only files that were being written)
The issue seems to happen randomly, some files would stream normally, and some rclone would download completely before playing.
I installed rclone from the default software manager in linux mint.
Run the command 'rclone version' and share the full output of the command.
rclone v1.50.2
os/arch: linux/amd64
go version: go1.13.6
Which cloud storage system are you using? (eg Google Drive)
Google Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone mount "Unlimited Drive:" /home/truck/unlimited --vfs-cache-mode writes --cache-dir "/home/truck/Rclone Cache"
The rclone config contents with secrets removed.
It's encrypted
A log from the command with the -vv
flag
2022/02/18 06:51:36 DEBUG : rclone: Version "v1.50.2" starting with parameters ["rclone" "mount" "Unlimited Drive:" "/home/truck/unlimited" "--vfs-cache-mode" "writes" "--cache-dir" "/home/truck/Rclone Cache" "-vv"]
asdffdsa
(jojothehumanmonkey)
February 18, 2022, 1:39am
2
hello and welcome to the forum,
this is not a rclone bug
--- update to latest stable rclone v1.57.0
the only way to get the latest rclone is
https://rclone.org/downloads/#script-download-and-install
--- change
--vfs-cache-mode writes
to
--vfs-cache-mode full
would be helpful to post the redacted rclone config
htusern84
(Ben Dover)
February 18, 2022, 1:41am
3
I do not wish to use full cache. I'm trying to cache only writes but it's caching everything (that too randomly). How is that not a bug?
And my config file is encrypted so its not really possible for me to post it.
asdffdsa
(jojothehumanmonkey)
February 18, 2022, 1:49am
4
htusern84:
How is that not a bug?
well, so far there is zero evidence of a rclone bug.
-- redacted config file not posted
--- debug log, not posted
-- using a very old version of rclone, before v1.53.0
.
htusern84
(Ben Dover)
February 18, 2022, 1:49am
5
I have upgraded to the latest version and its still caching fully instead of only caching "writes"
truck@linux-mint:~$ rclone version
rclone v1.57.0
os/version: linuxmint 20.3 (64 bit)
os/kernel: 5.4.0-100-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.17.2
go/linking: static
go/tags: none
htusern84
(Ben Dover)
February 18, 2022, 1:51am
6
Okay help me out here, the config file is encrypted, how do I decrypt it for my viewing in order to post it here.
And I did post the debug log as I was asked to while making the thread
asdffdsa
(jojothehumanmonkey)
February 18, 2022, 1:58am
7
that debug log was for the many years old version of rclone, so nothing of use there.
if you are going to call this a rclone bug, then need a full debug log documenting that.
so far, not providing any actionable info??
as per the docs, for --vfs-cache-mode writes
"read/write files are buffered to disk first"
htusern84
(Ben Dover)
February 18, 2022, 2:02am
8
Not really trying to "post" time, genuinely facing an issue and made a thread so I could perhaps get help on it.
htusern84
(Ben Dover)
February 18, 2022, 2:03am
9
if you are going to call this a rclone bug, then need a full debug log documenting that.
Not really calling it anything, i faced an issue and I thought the forums would be a good place to get help.
Fwiw, you could be a bit less toxic
htusern84
(Ben Dover)
February 18, 2022, 2:05am
10
truck@linux-mint:~$ rclone mount "Unlimited Drive:" /home/truck/unlimited --vfs-cache-mode writes --cache-dir "/home/truck/Rclone Cache" -vv
2022/02/18 07:28:04 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "mount" "Unlimited Drive:" "/home/truck/unlimited" "--vfs-cache-mode" "writes" "--cache-dir" "/home/truck/Rclone Cache" "-vv"]
2022/02/18 07:28:04 DEBUG : Creating backend with remote "Unlimited Drive:"
htusern84
(Ben Dover)
February 18, 2022, 2:10am
11
The file in question is a video file, read only, I'm not writing anything to it.
htusern84
(Ben Dover)
February 18, 2022, 2:22am
13
[Backup]
type = drive
client_id = redacted
client_secret = redacted
scope = drive
token = {"access_token":"redacted","expiry":"2022-02-18T07:56:08.14166618+05:30"}
team_drive =
root_folder_id = redacted
[Unlimited Drive]
type = crypt
remote = Backup:Private
password = redacted
password2 = redacted
[bx1]
type = box
token = {"access_token":"redacted","token_type":"bearer","refresh_token":"redacted","expiry":"2021-09-13T07:36:02.2051576+05:30"}
[Box 1]
type = crypt
remote = bx1:Private
password = redacted
password2 = redacted
filename_encryption = standard
directory_name_encryption = true
[rd1]
type = drive
client_id = redacted
client_secret = redacted
scope = drive
token = {"access_token":"redacted"}
team_drive =
[rd1e]
type = crypt
remote = rd1:Private
password = redacted
password2 = redacted
[rd2]
type = drive
client_id = redacted
client_secret = redacted
scope = drive
token = {"access_token":"redacted"}
team_drive =
[rd2e]
type = crypt
remote = rd2:Private
password = redacted
password2 = redacted
[PUnion]
type = union
upstreams = rd1e: rd2e:
Config file with all sensitive information removed
htusern84
(Ben Dover)
February 18, 2022, 3:22am
14
Heres the full log
as you can see its saving the whole file even though i didnt set it to full.
It's hard to tell what you've done as your cache-dir is all jacked up probably from the different versions.
Your debug:
2022/02/18 08:42:01 DEBUG : Test folder/17.mp4: vfs cache: removed metadata from cache as cache file doesn't exist
But then:
2022/02/18 08:42:01 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Test folder/17.mp4 not removed, freed 0 bytes
The read you are asking about comes from that local file that's already there so this line tells me it wasn't put there natively through rclone while the cache was there:
2022/02/18 08:42:01 INFO : vfs cache: cleaned: objects 9 (was 9) in use 0, to upload 0, uploading 0, total size 0 (was 0)
Since there is nothing upload or your whole metadir is messed up.
Stop rclone
Delete/Move the cache-dir (delete if you have no data that's out value)
Start rclone
Repeat your process
Share a new complete debug log
htusern84
(Ben Dover)
February 18, 2022, 1:58pm
16
I have repeated the process with a new cache directory.
As you can see, even though the file is being opened for read-only, rclone is saving it in disk.
Test 2 with different files and a clean cache directory
This time it did not save the files.
asdffdsa
(jojothehumanmonkey)
February 18, 2022, 2:10pm
17
sorry if that came across but imho, i think i was most helpful.
i wrote "hello and welcome to the forum,"
let you know this is not a rclone bug.
let you know that you were using a very old version of rclone,
prior to v1.53.0
which had a major update to the vfs file cache.
tried twice to get you to post a full debug log, as that would show what the problem is.
the forum moderator overloads will take down topics missing basic info such as config file and debug log, was trying to prevent that.
made no personal comments and did not use any adjectives about you.
The issue whatever you app your using is opening it read/write:
2022/02/18 19:11:06 DEBUG : Test folder/18.mp4: Open: flags=OpenReadWrite
and not read only so it's being saved.
A normal read open like this:
2022/02/18 09:41:21 DEBUG : jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv: Attr:
2022/02/18 09:41:21 DEBUG : jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv: >Attr: a=valid=1s ino=0 size=404291584 mode=-rw-rw-r--, err=<nil>
2022/02/18 09:41:22 DEBUG : jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv: Open: flags=OpenReadOnly
2022/02/18 09:41:22 DEBUG : jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv: Open: flags=O_RDONLY
Would not write the file to disk so it's doing exactly what you've asked it to.
In checking the second log, you can see it has no writes, but only read:
grep -i write mrQbx33m.txt
truck@linux-mint:~$ rclone mount "Unlimited Drive:" /home/truck/unlimited --vfs-cache-mode writes --cache-dir "/home/truck/Rclone Cache 3" -vv
2022/02/18 19:21:20 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "mount" "Unlimited Drive:" "/home/truck/unlimited" "--vfs-cache-mode" "writes" "--cache-dir" "/home/truck/Rclone Cache 3" "-vv"]
2022/02/18 19:23:53 DEBUG : rclone: Version "v1.57.0" finishing with parameters ["rclone" "mount" "Unlimited Drive:" "/home/truck/unlimited" "--vfs-cache-mode" "writes" "--cache-dir" "/home/truck/Rclone Cache 3" "-vv"]
and
grep -i ONLY mrQbx33m.txt
2022/02/18 19:21:56 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:21:56 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:21:57 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:21:57 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:21:59 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:21:59 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:00 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:00 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:01 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:01 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:03 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:03 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:27 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv not removed, freed 0 bytes
2022/02/18 19:22:44 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:44 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:46 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:46 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:48 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:48 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:49 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:49 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:50 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:50 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:22:53 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=OpenReadOnly+OpenNonblock
2022/02/18 19:22:53 DEBUG : Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv: Open: flags=O_RDONLY|0x800
2022/02/18 19:23:27 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E01-Tennis Club [91826C00].mkv not removed, freed 0 bytes
2022/02/18 19:23:27 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item Anime/The Tatami Galaxy S01 1080p BDRip 10 bits DD x265-EMBER/S01E02-Movie Club [77805C71].mkv not removed, freed 0 bytes
So it's only read only and that means no items saved to the cache area.
htusern84
(Ben Dover)
February 18, 2022, 4:10pm
20
Not an issue with rclone then. I was playing the video with VLC, I guess vlc is opening it in read/write . Any idea how I can prevent that?
You can't.
It's however the app is written and how they decide to open it. The change would be in the application.