this is full debug, i cant passte all its too long, but its cache related
well, did not see Fatal error: mount not ready
and
no idea of the exact command that was run as seen in the debug log.
I uploaded a lot of videos and now i figure out that all video was only saved in cache, not uploaded on Dropbox.
now when i run cmd with -vv it never finish and i think now its uploading videos from cache to dropbox,
is there anyway when i upload any new video it also upload on mounted dropbox?
also can see a msg that transfer exceeded, i think rclone reach the transfer limit,
any idea?
so now it is uploading the files to dropbox or not?
this could be the problem.
DEBUG : video/2022/01/19/02b89e72683fe500b78edd81049b2742/576p_312.ts: vfs cache: delaying writeback as --transfers exceeded
as to the solution, not sure,
the dropbox documentation for batch features is very complicated and i have no experience.
i took a look at @Animosity022 systemd files and there are no batch tweaks done, so it would seem that the defaults settings work.
I deleted everything and made a new rclone mount, and uploaded 100gb data, only 3.5gb uploaded on Dropbox,
will upload on Dropbox later or what?
my mount cmd is below
rclone mount Dropbox: /www/wwwroot/videos/box --daemon --allow-other --poll-interval 10s --tpslimit 12 --tpslimit-burst 12 --dir-cache-time 150h --vfs-cache-mode full --vfs-cache-max-age 150h --vfs-cache-max-size 100G --cache-dir /www/wwwroot/cache
now im experimenting what will happened when cache directory will reach its limit,
someone who are using dropbox please guide me, is there any upload limit on dropbox?
Thanks
any help @Animosity022 @asdffdsa ?
mounted remote not transferring data from remote to dropbox
Thanks
@asdffdsa i restarted server and now again cannot mount dropbox, with -vv getting a very long debug that never finish, a sample is below
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1151.ts: vfs cache: delaying writeback as --transfers exceeded
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_521.ts: vfs cache: truncate to size=420368
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_521.ts: vfs cache: setting modification time to 2022-01-20 17:33:39.688288055 +0100 CET
2022/01/20 23:51:23 INFO : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_521.ts: vfs cache: queuing for upload in 5s
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73: Added virtual directory entry vAddFile: "576p_521.ts"
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1151.ts: vfs cache: delaying writeback as --transfers exceeded
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1149.ts: Uploading chunk 2/1
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_115.ts: Uploading chunk 2/1
2022/01/20 23:51:23 DEBUG : Dropbox root '': Checking for changes on remote
^C2022/01/20 23:51:23 INFO : Signal received: interrupt
2022/01/20 23:51:23 INFO : Dropbox root '': Commiting uploads - please wait...
2022/01/20 23:51:23 DEBUG : Dropbox root '': Committing sync batch length 1 starting with: /video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1148.ts
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_522.ts: vfs cache: truncate to size=453080
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_522.ts: vfs cache: setting modification time to 2022-01-20 17:33:40.04429315 +0100 CET
2022/01/20 23:51:23 INFO : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_522.ts: vfs cache: queuing for upload in 5s
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73: Added virtual directory entry vAddFile: "576p_522.ts"
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1151.ts: vfs cache: delaying writeback as --transfers exceeded
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1150.ts: Uploading chunk 1/1
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_523.ts: vfs cache: truncate to size=730756
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_523.ts: vfs cache: setting modification time to 2022-01-20 17:33:40.448298926 +0100 CET
2022/01/20 23:51:23 INFO : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_523.ts: vfs cache: queuing for upload in 5s
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73: Added virtual directory entry vAddFile: "576p_523.ts"
2022/01/20 23:51:23 DEBUG : video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1151.ts: vfs cache: delaying writeback as --transfers exceeded
Thanks
it looks like rclone is uploading files, for example,
video/2022/01/20/0160f35c0e770639bf50b1ac846dae73/576p_1150.ts: Uploading chunk 1/1
now, check dropbox to confirm that rclone is uploading files, any files.
i leave it running, lets see when it will finish, but dont you think this is strange why not rclone uploaded these files before when its mounted?
i will update you later when its finish
thanks a lot
sorry, not understanding what you wrote.
the question is "has rclone uploaded any files at all, uploaded at least one file?
yes or no?
i did some research,
i would kill the mount, add --transfers=32 and re-start the mount command.
1 Like
yes rclone is uploading files on dropbox, i can see recent added files,
process is very slow but it keep uploading on dropbox
next time i will add this in mount cmd,
Thanks
that is because by default, rclone transfers four files at a time, but the rclone docs for dropbox suggest
--transfers parameter (32 or 64 works fine)
there is nothing wrong with killing mount, add --transfers=32 and re-starting the mount.
that way, the upload would finish much sooner.
but that is up to you.
1 Like
oh ok great i will kil and try to mount with modified cmd, i have one doudt how to add this transfer parameter, plz chack below
--transfers=32
or
--transfers 32
Thanks
that is documented at https://rclone.org/docs/#options
Options that take parameters can have the values passed in two ways,
`--option=value` or `--option value`
1 Like
i leave running my pc with -vv cmd and it was uploading files like 10 hours and now i can see this debud and cmd is stop by itself
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/02b89e72683fe500b78edd81049b2742/576p_734.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/3b3f2eb2729c9369a42a8162464b5442/576p_1681.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/7b2ca982feb1bc49459a5b3aa4badb2c/576p_1121.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/85b4a78936e3a9d1ad09ddd5c257ecf8/576p_914.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/8bb3c209c3e89b5dfba240671603b70f/360p_025.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/9986e44893cb52b8276c8a08f7ae0e86/576p_721.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/ec3d5a061b0b98e100e9650cda8ee2c1/576p_1542.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/8629e5baa17d11ebaa7acdfa918e8635/576p_1062.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/af378b92fe61347b5c504784f5a16d9e/576p_1183.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/b8f7ce9d2e4932b6c554a8c93b441966/576p_951.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/bb4988f5de55901772c2a34f53928296/576p_1607.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/d2691891252c7ea9af20c64507c1605d/576p_923.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge=540000000000000, emptyOnly=false): item video/2022/01/20/dd4cd975e00a9ca8a0681f4fa66b851f/576p_1531.ts not removed, freed 0 bytes
2022/01/21 12:02:09 DEBUG : vfs cache RemoveNotInUse (maxAge
and i can see thar dropbox is mounted,
Thanks
@asdffdsa Thank you soo much, things are now working properly, i have good server so im using
--transfers 64
btw im using
--dir-cache-time 150h
i think this is very high entry, what you suggest?
i tried a lot to find out in doc about
--tpslimit 12 --tpslimit-burst 12
but no success, i almost understand all other entries except these two, can you link any doc plz?
i really appreciated your patience and willing to help other, you are great person, Thanks again and wish u a good luck in ur life
that caches the vfs dir cache, not the vfs file cache.
given that dropbox is a polling remote, i would use `--dir-cache-time 5000h
those settings are documented.
those values have been battle-tested by other rcloners using dropbox.
so just use them as is.
live long and propser
to better understand the differences between
--- vfs dir cache
--- vfs file cache
take a read of something i wrote
Status about using Rclone for music storage / playback in 2021. Access times improved? - #34 by asdffdsa
Thanks im clear with these things,
now started another issue, i cannot play videos that are not in rclone cache, how to fix this issue?
Thanks
well, i never use a vfs file cache , in effect --vfs-cache-mode=off and many other rcloners do the same.
we can stream videos, no problem.