I am prototyping a systems design and it's looking very nice with a SFTP server (A) and another server (B) that rclone presents the SFTP as a volume to the docker containers. Seems to work well until I kill the SFTP and instead of getting a timeout or something on the file IO the terminal hangs... it will come back about two minutes or so after I restore the SFTP.
What I would like to do is have it timeout within a second and then in my application in the docker I will go to another Rclone docker volume that contains a backup location on B2 to look for the files.
Is there a way on the Rclone side or on the docker container side to timeout the IO?
What is your rclone version (output from rclone version)
os/version: ubuntu 20.04 (64 bit)
os/kernel: 5.4.0-80-generic (x86_64)
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Interesting, there is some kind of retry happening somewhere as when I reconnect server A while doing a transfer inside the container it eventually sees it and resumes the transfer after maybe 4 minutes.
I also note that this works for the rclone mount: rclone mount node3sftp: /mnt/2 --daemon-timeout=1s
The --daemon-timeout name is misleading. It would be better named --macos-mount-kernel-timeout but it's called what it's called, we'll keep the name for compatibility. Actually this option works only on MacOS and sets timeout in the kernel mount handler. It's useless on other systems.
I'm not a guru of kernel mount APIs but AFAIK the FUSE API does not provide the mount provider with much means to signal a critical condition or ask to terminate. rclone is left to retry forever on all errors, as soon as mount is established. All you can do is "docker kill" your container.
Ah! This works for the Rclone mount giving an I/O error and also answering future file requests when the source returns.
This controls the number of low level retries rclone does.
A low level retry is used to retry a failing operation - typically one HTTP request. This might be uploading a chunk of a big file for example. You will see low level retries in the log with the -v flag.
This shouldn't need to be changed from the default in normal operations. However, if you get a lot of low level retries you may wish to reduce the value so rclone moves on to a high level retry (see the --retries flag) quicker.
Disable low level retries with --low-level-retries 1.
Is there any way to support the same in the Docker volume driver?