Do you get any ERRORs in the log file?
There are no errors in the logs. When "ls" the nzbget folder, it just freezes the terminal AND nzbget web gui.
Also what I found is that I have to set "--vfs-cache-mode=writes" to write files on local folders. Is this by design?
Does it ever come back?
That is the VFS layer which is separate from the union backend. Yes you'll need that for various things.
multi write union not working (latest beta). Individual remote working fine but when wrapped with Union, it does not work. Search works (lsf, ls) but write/create not working on Union. (onedrive)
[samsungod]
type = union
upstreams = od2:SamsungDrive od3:SamsungDrive
Log
harry@ubuntu16:~$ rclone mkdir samsungod:testdir -vv
2020/03/31 11:58:01 DEBUG : rclone: Version "v1.51.0-138-g5f9be3dd-beta" starting with parameters ["rclone" "mkdir" "samsungod:testdir" "-vv"]
2020/03/31 11:58:01 DEBUG : Using config file from "/home/harry/.config/rclone/rclone.conf"
2020/03/31 11:58:11 DEBUG : union root 'testdir': Making directory
2020/03/31 11:58:14 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2020/03/31 11:58:14 DEBUG : union root 'testdir': Making directory
2020/03/31 11:58:15 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2020/03/31 11:58:15 DEBUG : union root 'testdir': Making directory
2020/03/31 11:58:15 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2020/03/31 11:58:15 DEBUG : 10 go routines active
2020/03/31 11:58:15 Failed to mkdir: object not found
harry@ubuntu16:~$
lsf or rmdir works
harry@ubuntu16:~$ rclone lsf samsungod:
generatepress_child.zip
harry@ubuntu16:~$
Is it just mkdir that is failing?
I tried a similar test and it worked for me.
Does rclone mkdir od3:SamsungDrive/testdir work for you?
mkdir copy sync all fails. But ls lsf lsd delete rmdir works.
Yes. rclone mkdir od3:SamsungDrive/testdir and rclone mkdir od2:SamsungDrive/testdir both works.
What --dump parameters will need for you to debug?
Strange... I can't replicate this even using two onedrives.
Did you create the directories od3:SamsungDrive and od2:SamsungDrive ?
A log with -vv --dump bodies will give some idea what is going on but will be might confusing too! If you could do that on the rclone mkdir that would be perfect.
Yes. I manually created those directories first using Rclone.
harry@ubuntu16:~$ rclone lsd od2:
-1 2020-03-21 16:36:19 0 SamsungDrive
harry@ubuntu16:~$ rclone lsd od3:
-1 2020-03-21 16:36:25 0 SamsungDrive
Thanks for that. It doesn't appear to try to create a directory, it is not getting that far. @Max-Sum can you see what is going on?
Hey, been playing with the new multiwrite union, looks fantastic and can replace mergerfs for me.
That said I'm struggling to get roughly the same experience with this mergerfs command:
mergerfs $HOME/mounts/media-local:$HOME/mounts/media-remote=NC $HOME/mounts/media -o rw,atomic_o_trunc,big_writes,splice_move,splice_read,splice_write,async_read=false,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true,ignorepponrename=true,moveonenospc=true
In this example I have three dirs, media, media-local and media-remote. First is the union mount, second is the local tree, and finally the mount crypt gdrive. This works great, can add files to media folder, they're written to the media-local folder and an hourly script uploads them to the google drive remote.
However when I tried something similar with this union backend I can see the google drive in media, media-local can be written to but those files are not visible in the union mount.
I suspect this will be a common usecase so some steering on how to configure it to be similar to the above mergerfs command string will be super helpful!
Can you post your best effort at a config?
I have created a union consisting of an existing GSuite remote folder set to read-only and a local folder. All other settings have been set to their default values.
Both remotes are holding an identical folder structure.
The remote has been mounted as a drive by running it as a service with nssm.
Everything shows up as expected and file operations within the several folders appear to work properly, with one exception so far:
When i try to write data to the union root folder (the level which shows all the available sub-folders), Windows won't allow me to do so.
I have not been able to find get around this or find the reason for this behaviour. It might be clear that writing to the remote won't be possible because of the :ro setting but at least writing to the local folder root should be possible from my understanding.
Hopefully this explenation was clear enough for someone to help investigate this issue.
Oh, sorry for not providing this:
rclone version:
rclone v1.51.0-161-g2c4aadb5-beta
- os/arch: windows/amd64
- go version: go1.14.2
remote configuration:
[tv]
type = union
upstreams = gdrive_crypt:/Serien:ro d:\Downloads\out\tv
mount command:
mount tv:/ m: --allow-other --dir-cache-time 96h --drive-chunk-size 32M --log-level INFO --timeout 1h
I tried it on Linux and it works as expected - writing to the root writes to the local disk.
Is it just the root you can't write to? Can you write to subfolders? Or create folders?
Can you try using rclone copy -vv to copy a file into the root and paste the log output?
Yes, it only affects the root folder.
This is the output from the copy command:
2020/04/15 18:04:06 DEBUG : rclone: Version "v1.51.0-161-g2c4aadb5-beta" starting with parameters ["rclone" "copy" "c:\Tools\rclone\README.txt" "tv:/" "-vv" "--config" "rclone.conf" "--log-file" "rctest.log"]
2020/04/15 18:04:06 DEBUG : Using config file from "C:\Tools\rclone\rclone.conf"
2020/04/15 18:04:08 DEBUG : README.txt: Need to transfer - File not found at Destination
2020/04/15 18:04:08 ERROR : README.txt: Failed to copy: object not found
2020/04/15 18:04:08 ERROR : Attempt 1/3 failed with 1 errors and: object not found
2020/04/15 18:04:08 DEBUG : README.txt: Need to transfer - File not found at Destination
2020/04/15 18:04:08 ERROR : README.txt: Failed to copy: object not found
2020/04/15 18:04:08 ERROR : Attempt 2/3 failed with 1 errors and: object not found
2020/04/15 18:04:09 DEBUG : README.txt: Need to transfer - File not found at Destination
2020/04/15 18:04:10 ERROR : README.txt: Failed to copy: object not found
2020/04/15 18:04:10 ERROR : Attempt 3/3 failed with 1 errors and: object not found
2020/04/15 18:04:10 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 1.3s
2020/04/15 18:04:10 DEBUG : 6 go routines active
2020/04/15 18:04:10 Failed to copy: object not found
Hmm! What if you try running the mount with --vfs-cache-mode writes does that help?
It really seems to make a difference. I was able to write the test file now. I will do some further testing.
Anyway, thanks a lot!
1 Like
I am now having the effect that changes to either the local or the remote part of the union are not visible on the mounted union. For example, when i add files to the local directory, they don't show up when browsing the union. I first have to re-mount by restarting the service. I guess there is a flag that will control this behaviour but cannot figure out which one it could be.
The problem is that the directories are cached for --dir-cache-time by default and that unlike writing to google drive, writing to the local disk doesn't have a change notify service so these changes won't get noticed immediately.
Using vfs/refresh via the rc should fix your problem or you can reduce --dir-cache-time.