This is the bug I'm referring to.
Since Nick was wondering about whether the bug was fixed by linux kernel changes or rclone changes, I just wanted to confirm that it was fixed by the rclone beta changes, and not by the kernel.
Explanation:
I was still affected by the bug while on Kernel 5.4.6 and the latest rclone version on the Arch repositories (v1.50.2, go1.13.4).
After installing the latest beta (v1.50.2-093-gf754d897-beta, go1.13.5), the bug was fixed.
Hope this helps!
Thanks for testing that.
I'm glad it is fixed in the beta!
I'm not sure exactly what fixed it though
I've just built rclone 1.50.2 from source with go compiler 1.13.5, and the bug persists, so the fix must be due to rclone code or library change.
After playing around with git bisect, it looks like the bug was fixed by commit 21c6babdbb0e8e1f7fb4633340fd6ebd8be366d3.
Thanks for doing that - very interesting result! There were a lot of fuse changes between Linux 5.3 and 5.4. I don't know exactly what they are all about but I see a lot of moving to new API.
So I wonder if the new API for fuse is slower for sync access but fast for async access? If so then I expect other fuse filing systems will have noticed this.
$ git log --oneline v5.3..v5.4 -- fs/fuse | cat
091d1a726772 fuse: redundant get_fuse_inode() calls in fuse_writepages_fill()
e4648309b85a fuse: truncate pending writes on O_TRUNC
b24e7598db62 fuse: flush dirty data/metadata before non-truncate setattr
80da5a809d19 virtiofs: Remove set but not used variable 'fc'
a9bfd9dd3417 virtiofs: Retry request submission from worker context
c17ea0096103 virtiofs: Count pending forgets as in_flight forgets
5dbe190f3412 virtiofs: Set FR_SENT flag only after request has been sent
7ee1e2e631db virtiofs: No need to check fpq->connected state
51fecdd2555b virtiofs: Do not end request in submission context
6c26f71759a6 fuse: don't advise readdirplus for negative lookup
2b319d1f6f92 fuse: don't dereference req->args on finished request
3f22c7467136 virtio-fs: don't show mount options
112e72373d1f virtio-fs: Change module name to virtiofs.ko
8f744bdee4fe Merge tag 'virtio-fs-5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse
5addcd5dbd8c fuse: Make fuse_args_to_req static
9ad09b1976c5 fuse: fix memleak in cuse_channel_open
e5854b1cdf6c fuse: fix beyond-end-of-page access in fuse_parse_cache()
0ed4059302a7 fuse: unexport fuse_put_request
dc69e98c241e fuse: kmemcg account fs data
30c6a23d34cb fuse: on 64-bit store time in d_fsdata directly
d5880c7a8620 fuse: fix missing unlock_page in fuse_writepage()
a62a8ef9d97d virtio-fs: add virtiofs filesystem
15c8e72e88e0 fuse: allow skipping control interface and forced unmount
783863d6476c fuse: dissociate DESTROY from fuseblk
8fab01064436 fuse: delete dentry if timeout is zero
0cd1eb9a4160 fuse: separate fuse device allocation and installation in fuse_conn
ae3aad77f46f fuse: add fuse_iqueue_ops callbacks
0cc2656cdb0b fuse: extract fuse_fill_super_common()
4388c5aac4ba fuse: export fuse_dequeue_forget() function
79d96efffda7 fuse: export fuse_get_unique()
95a84cdb11c2 fuse: export fuse_send_init_request()
14d46d7abc39 fuse: export fuse_len_args()
04ec5af0776e fuse: export fuse_end_request()
f22f812d5ce7 fuse: fix request limit
05ea48cc2b09 fuse: stop copying pages to fuse_req
d49937749fef fuse: stop copying args to fuse_req
145b673bd208 fuse: clean up fuse_req
7213394c4e18 fuse: simplify request allocation
66abc3599c3c fuse: unexport request ops
75b399dda5be fuse: convert retrieve to simple api
4cb548666e4c fuse: convert release to simple api
b50ef7c52ad7 cuse: convert init to simple api
615047eff108 fuse: convert init to simple api
33826ebbbe4b fuse: convert writepages to simple api
43f5098eb82b fuse: convert readdir to simple api
134831e36bbd fuse: convert readpages to simple api
45ac96ed7c36 fuse: convert direct_io to simple api
1259728731a7 fuse: add simple background helper
338f2e3f3341 fuse: convert sync write to simple api
00793ca5d443 fuse: covert readpage to simple api
a0d45d84f4c9 fuse: fuse_short_read(): don't take fuse_req as argument
093f38a2c1a8 fuse: convert ioctl to simple api
4c4f03f78ca9 fuse: move page alloc
4c29afece872 fuse: convert readlink to simple api
68583165f962 fuse: add pages to fuse_args
1ccd1ea24962 fuse: convert destroy to simple api
e413754b267e fuse: add nocreds to fuse_args
3545fe211282 fuse: convert fuse_force_forget() to simple api
454a7613f54e fuse: add noreply to fuse_args
c500ebaa908d fuse: convert flush to simple api
40ac7ab2d021 fuse: simplify 'nofail' request
1f4e9d03d1fb fuse: rearrange and resize fuse_args fields
d5b4854357f4 fuse: flatten 'struct fuse_args'
76e43c8ccaa3 fuse: fix deadlock with aio poll and fuse_iqueue::waitq.lock
c7eb6869632a vfs: subtype handling moved to fuse
c30da2e981a7 fuse: convert to use the new mount API
56d250ef9650 cuse: fix broken release
17b2cbe29492 fuse: cleanup fuse_wait_on_page_writeback
1fb027d75964 fuse: require /dev/fuse reads to have enough buffer capacity (take 2)
On Fedora it's still the same especially with thunar it fresh to death if I open the mounted onedrive folder. I use rclone from fedora repo with thelastest 1.50.2 with go 1.13.5 on x64
Try the latest beta of rclone...
I did, on https://beta.rclone.org/v1.50.2-200-gf80d9855-beta/ should be the Beta right? Still slow and It's unbreable, only works on CLI using LS command or VIM
Tried and result always on this
[ben@thinkpad OneDrive]$ cd ..
[ben@thinkpad ~]$ sudo umount OneDrive/
[ben@thinkpad ~]$ rclone mount --vfs-cache-mode writes onedrive: ~/OneDrive --buffer-size 1G --dir-cache-time=1000h
[ben@thinkpad ~]$ rclone mount --vfs-cache-mode writes onedrive: ~/OneDrive --buffer-size 1G --dir-cache-time=1000h
2020/02/01 00:18:53 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/02/01 00:18:54 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 2/10: unexpected EOF
2020/02/01 00:18:55 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 3/10: unexpected EOF
2020/02/01 00:18:56 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 4/10: unexpected EOF
2020/02/01 00:18:57 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 5/10: unexpected EOF
2020/02/01 00:18:57 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 6/10: unexpected EOF
2020/02/01 00:18:58 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 7/10: unexpected EOF
2020/02/01 00:18:58 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 8/10: unexpected EOF
2020/02/01 00:18:59 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 9/10: unexpected EOF
2020/02/01 00:19:01 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 10/10: unexpected EOF
2020/02/01 00:19:01 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: unexpected EOF
2020/02/01 00:19:01 ERROR : Documents/Baksos/Catatan PerkabTrans.docx: ReadFileHandle.Read error: low level retry 1/10: EOF
2020/02/01 00:19:05 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 1/10: unexpected EOF
2020/02/01 00:19:05 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 2/10: unexpected EOF
2020/02/01 00:19:07 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 3/10: unexpected EOF
2020/02/01 00:19:08 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 4/10: unexpected EOF
2020/02/01 00:19:08 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 5/10: unexpected EOF
2020/02/01 00:19:09 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 6/10: unexpected EOF
2020/02/01 00:19:10 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 7/10: unexpected EOF
2020/02/01 00:19:10 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 8/10: unexpected EOF
2020/02/01 00:19:11 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 9/10: unexpected EOF
2020/02/01 00:19:12 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 10/10: unexpected EOF
2020/02/01 00:19:12 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: unexpected EOF
2020/02/01 00:19:12 ERROR : Documents/Baksos/LPJ Perkab Trans Baksos 2016.docx: ReadFileHandle.Read error: low level retry 1/10: EOF
You'd need to include a bit more and a debug log as that really doesn't say much other than it can't read a file.
Why are you using a 1G buffer size?
Some people said it help with the file that have downloaded, the list file so I use it anyway.
ehm is there any docs about run it and show verbose level ? I'm quite new to rclone, pardon for intrusion.
On Thunar, it will slow down when I click on the file(not opening it, just clicking) and I just guess that it's because thunar downloading the file description and it need time, but for folder it's very slow.
On PCMANFM, it will load forever
You can add "-vv" and if you want use "--log-file /tmp/rclone.og" or something along those lines.
No idea what that is.
I hope this log help. PCMANFM is file manager of lxde, on thunar it's still loading but on pcmanfm it load forever.
Oh no, It's OneDrive sir. Now waiting for new kernel, but seems it's stil slugish on 5.5.0 vanilla kernel fedora. I don't know if it's network issue or not, but my connection is pretty fast.
Is there no cache for file list and file attribute on rclone mount?
Onedrive and MS docs behaves strangely sometimes... Can you try with a non MS doc file?
The VFS layer has directory and attribute caching
--dir-cache-time duration Time to cache directory entries for. (default 5m0s)just to add a data point, kernel 5.5.1 + rclone v1..51 does not have any mount speed problems
Ok. I already use that, on v1.51 and kernel 5.4.15-200.fc31.x86_64 on fedora seems much better, not fast, but better than before without too much lag but still lag. Thank you. But seems not seemlessly like old version.
new error raised
2020/02/05 23:30:13 ERROR : Pictures/Beasiswa/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:30:13 ERROR : Pictures/Beasiswa/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:34 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:34 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:34 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:34 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:34 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:35 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:39 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:39 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:39 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:41 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:41 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:41 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:41 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:55 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS
2020/02/05 23:31:55 ERROR : Documents/Silabus/: Dir.Stat error: couldn't list files: InvalidAuthenticationToken: Unable to initialize RPS