Config not read from fifo anymore

Suspected Bug
1

What is the problem you are having with rclone?

Reading config file from fifo (see example command) worked up to version 1.54.1. Since 1.55.0 rclone stops with error message. (Reading fifo is a nice feature, it would be great if it was possible to make it work again)

What is your rclone version (output from rclone version)

rclone v1.55.0

  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.2
  • go/linking: static
  • go/tags: cmount

Which OS you are using and how many bits (eg Windows 7, 64 bit)

CentOS 7, 64bit

Which cloud storage system are you using? (eg Google Drive)

Not relevant.

The command you were trying to run (eg rclone copy /tmp remote:tmp)

cat rclone.cfg > rc.fifo  &  sleep 1 &&  rclone  --config  rc.fifo  mount  local:/home/ /dev/shm/xxx

The rclone config contents with secrets removed.

[local]
type = local

A log from the command with the -vv flag

No log printed; program stops with error (`...' replaces a folder path):  
2021/05/23 22:34:21 Failed to load config file "/run/media/.../rc.fifo": seek /run/media/.../rc.fifo: illegal seek
2

Interesting!

I think this is the encrypted config support sniffing the config file to see if it is encrypted.

It is possible to pass config items via the command line easily enough - just wondering why you don't do that instead?

In 1.55 you can also pass config items as part of a connection string, eg

rclone lsd ":http,url='https://pub.rclone.org':"
rclone lsf ":http,url='https://example.com':path/to/dir"
rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir
rclone copy :sftp,host=example.com:path/to/dir /tmp/dir
3

Hello Nick,

Thank you for having a look at it. I think for the specific example I gave originally in the post (which I wanted to use to show my issue with the minimum complication) your solution would work. In my real situation I have crypted files and what I would like to achieve is that the config file is stored on another (remote) machine, separately from the data. So, when I need the config file, I could get it via ssh from the remote machine, immediately piped into the fifo which is then read out by rclone (also immediately). This way the config file is never stored on the local machine. Probably this cannot be achieved using a connection string.

Reading fifo used to work. I know when big changes happen to the code, e.g. switching to new libraries, some details of the behavior may change and it may not be straightforward to get some old features work again. I just wanted to mention the current issue (without knowing details of the code) so that if the fix is simple, it could be done. At the same time I also know sometimes the code structure does not allow to achieve "simple changes" in simple ways.

4

You are probably right!

Indeed...

If you wanted to help it would be really useful to know exactly which commit broke this feature. So build rclone from source then use git bisect to find the commit which introduced the problem.

5

OK, I am going to locate the commit where it appeared and report on the outcome here (sometime later this week). Thanks!

6

Thank you - that would be great :slight_smile:

7

Hello Nick,

I have downloaded the complied beta versions (linux-amd64) from https://beta.rclone.org/. Based on those binaries:

v1.55.0-beta.5256.6a9ae3201  and previous commits work
v1.55.0-beta.5257.4d8ef7bca  and later commits fail to read from fifo
8

I just copy here the link to the diff page.

9

That commit doesn't touch the config reading code at all so I'm suprised that makes a difference...

Can you check - does the working rclone and the failing one - were they compiled with the same go version (check rclone version)

10

Sorry, I have not been available. I will check these again, soon!

1 Like
11

I am replying soon...

12

Sorry, I have been inactive for a long time...

So, I wrote a short script to check the different versions. I am uploading the script (rclone-check.sh.txt (1.3 KB)) and the result of the script (log.txt (27.2 KB)).
The file log.txt clearly shows at which version the problem appeared.

(I included the latest release and the latest beta among the versions, too. The problem still exists).

13

Dear Nick,

The reply above should have gone to you.

14

Looks like this is the commit that broke it

commit 6a9ae3201232f5761fc7603ceb158c29637b2454
Author: Nick Craig-Wood <nick@craig-wood.com>
Date:   Thu Mar 11 16:20:31 2021 +0000

    config: split up main file more and move tests into correct packages
    
    This splits config.go into ui.go for the user interface functions and
    authorize.go for the implementation of `rclone authorize`.
    
    It also moves the tests into the correct places (including one from
    obscure which was in the wrong place).

( BTW if you want to do this again, learn about git bisect which can automate this process. There is a little script to help in bin/bisect-rclone.sh )

First things first can you open a new issue on Github about this.

Secondly would you like to have a go at fixing this? It certainly needs a test otherwise we'll just break it again by accident!

15

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.