zzzhhh
November 23, 2021, 3:04pm
1
The ticket are in this thread which is related to --daemon. Specifically:
https://github.com/rclone/rclone/issue/2618
https://github.com/rclone/rclone/issue/5664
Since I am experiencing the same problem when --daemon is used, I want to keep track of it so that once it's fixed, I can upgrade to the fixed version. But these two tickets are both "404". How can I track the progress of fixing this issue?
There is a typo in each of those links, it's issues not issue.
opened 12:01PM - 05 Oct 21 UTC
VFS / mount
thinking
OS: Unix
terminal
refactoring
### References
This ticket:
- supersedes (?) #2618
- draws ideas from https… ://github.com/rclone/rclone/issues/2968#issuecomment-716286668
- and discussion at https://github.com/rclone/rclone/issues/2968#issuecomment-716679470 till https://github.com/rclone/rclone/issues/2968#issuecomment-728309447
- is related to #4764
Also related to a number of [console password input](https://github.com/rclone/rclone/labels/terminal) tickets - see `Password prompts` in a comment below.
### What problem are you are trying to solve?
Looking at the rclone initialization code from the --daemon point of view I can see stuff that should run in the child only and not in parent:
- [rc-server](https://github.com/rclone/rclone/blob/v1.52.3/cmd/cmd.go#L379) - if enabled, it will bind to port in parent and cause "cannot bind" in daemonized child
- [NewFsDir](https://github.com/rclone/rclone/blob/v1.52.3/cmd/mountlib/mount.go#L305) - it chains to backend-specific, unconstrained, generally fork-unsafe [Fs.NewFs](https://github.com/rclone/rclone/blob/v1.52.3/fs/cache/cache.go#L90)
This calls for refactoring, esp. [initConfig](https://github.com/rclone/rclone/blob/v1.52.3/cmd/cmd.go#L362) can be split into a function common to all commands to be used in `cobra.OnInitialize` and into another function to contain child-only steps. Most commands would inherit it from `rootCmd.PersistentPreRun`. Daemonizing commands would disable it by `PersistentPreRun: func(){return nil}` and then call from child... just an idea
### How do you think rclone should be changed to solve that?
We might refactor backgrounding to have rughly 3 stages :
- _parent pre-init_ will run a hand-picked subset of fork-safe operations from current `cmd.initConfig`, before callsing`os.StartProcess`
- _child setup_ will take place in the child performing other unsafe operations from `initConfig` e.g. start rc-server, calls `fs.NewFs`, set up _vfs_, start fuse mount and return its status to parent.
This stage closely mimics current non-daemonizing flow.
At this stage child has fully working stdin/stdout/stderr (either parent redirects child streams through iself or perhaps the child just sits attached to the parent's terminal session) and can log debug/error messages to stderr or request user input from stdin.
Parent just pipes child's streams and waits for status or timeout.
- _child detach_ - parent receives daemonization/mount status from child (or detects timeout), logs it and exits, while child:
- finishes sending status to parent as a specially formatted line on stdout/stderr or via dedicated _pipe_,
- replaces stdin/stdout/stderr by _/dev/null_,
- stops output from stats and pacers,
- closes extra files and pipes,
- calls _setsid()_,
- probably does _setpgrp_ (detaches from process group) to ensure it will be later reaped by _pid 1_ when it exits or aborts,
- reopens log file descriptor and syslog connection if needed (because if any file descriptors remain from parent, the child later ends up as a zombie),
- probably detaches itself from original _cgroup_ (if it helps with systemd),
- proceeds with business as usual!
As a consequence, we have to:
- triage `initConfig` contents (it's rather small) between stages 1 and 2;
- think of switching child logger between file and redirected stdout(err);
- rethink config key;
- deactivate systemd log format in child;
- parent should issue warnings in advance if child's log file or stdin helper isn't configured.
- refactor all ReadLine, ReadPassword, fmt.Scanf and alikes in the core including _fs/config_ and in backends (where they usually read _2fa_ secrets) into a common wrapper that can:
- read terminal when available (and optionally hide user input if it's secret), or
- read stdin (prompt on stdout) piped from parent at stage 2, or
- use universally configured helper executables(files) when detached, or
- log errors when not configured;
Another idea is to keep the `StartDaemon` where it was already butuse a sort of sync mechanism between the parent process and the child process (pipe?).
### How to use GitHub
* Please use the 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to show that you are affected by the same issue.
* Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
* Subscribe to receive notifications on status change and new comments.
opened 05:43PM - 30 Sep 18 UTC
bug
duplicate
OS: Unix
API / rc
#### What is the problem you are having with rclone?
When using RC with rclone … mount and daemon mode, rclone starts with one process and then spawns a second process. Because the first process actually launches with the RC control the second process can not properly start RC. And the second process is the one that becomes the daemon.
#### What is your rclone version (output from `rclone version`)
1.43
#### Which OS you are using and how many bits (eg Windows 7, 64 bit)
UNRAID OS, Linux AMD 64Bit
#### Which cloud storage system are you using? (eg Google Drive)
Google Drive, Crypt, and Cache
#### The command you were trying to run (eg `rclone copy /tmp remote:tmp`)
```
rclone mount --fast-list --max-read-ahead 1024k --uid=99 --gid=100 --cache-tmp-upload-path=/mnt/user/rclone-upload-temp --cache-db-path=/mnt/user/rclone-cache --cache-chunk-path=/mnt/user/rclone-cache --syslog -v --daemon --rc --rc-addr=localhost:5578 --allow-other google-cryptic: /mnt/disks/google
```
#### A log from the command with the `-vv` flag (eg output from `rclone -vv copy /tmp remote:tmp`)
Unable to provide atm.
I fixed both links on the source side as well.
system
December 23, 2021, 3:10pm
4
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.