`SSH agent requested but SSH_AUTH_SOCK not-specified` error but ONLY in cron job

Help and Support
1

What is the problem you are having with rclone?

Getting error message
Failed to create file system for "QNap3:UpFlags/": couldn't connect to ssh-agent: SSH agent requested but SSH_AUTH_SOCK not-specified
when running any rclone command against that QNap3: remote in a cron job. WHen run interactively it works fine.

Run the command 'rclone version' and share the full output of the command.

rclone v1.63.1
- os/version: debian bookworm/sid (64 bit)
- os/kernel: 6.4.6-76060406-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yup!

Which cloud storage system are you using? (eg Google Drive)

A QNap on my LAN, over SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

It fails for all of these, with the same error:

rclone touch QNap3:UpFlags/dbkups.writing
rclone sync -v /media/david/Seag8/davidbkups/ QNap3:Backups/davidbkups/ 
rclone delete QNap3:UpFlags/dbkups.writing
rclone --sftp-skip-links size QNap3:Backups/davidbkups/

The rclone config contents with secrets removed.

[QNap3]
type = sftp
host = frier-nas-03.home.arpa
key_file = /home/david/.ssh/[redacted]
key_use_agent = true
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the -vv flag

When I run the script interactively from the command line, as david, the sync command produces:

2023/08/27 03:02:56 DEBUG : Setting --transfers "6" from environment variable RCLONE_TRANSFERS="6"
2023/08/27 03:02:56 DEBUG : Setting --check-first "true" from environment variable RCLONE_CHECK_FIRST="true"
2023/08/27 03:02:56 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "sync" "-vv" "/media/david/Seag8/davidbkups/" "QNap3:Backups/davidbkups/"]
2023/08/27 03:02:56 DEBUG : Creating backend with remote "/media/david/Seag8/davidbkups/"
2023/08/27 03:02:56 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2023/08/27 03:02:56 DEBUG : fs cache: renaming cache item "/media/david/Seag8/davidbkups/" to be canonical "/media/david/Seag8/davidbkups"
2023/08/27 03:02:56 DEBUG : Creating backend with remote "QNap3:Backups/davidbkups/"
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: New connection 10.25.1.30:54208->10.25.1.14:22 to "SSH-2.0-OpenSSH_8.0"
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Shell type "unix" from config
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Relative path resolved to "/share/CACHEDEV1_DATA/homes/david/Backups/davidbkups"
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Using root directory "/share/CACHEDEV1_DATA/homes/david/Backups/davidbkups"
2023/08/27 03:02:56 INFO  : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Running all checks before starting transfers
[... almost 6500 more lines exactly like those two ...]
2023/08/27 03:02:56 INFO  : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Checks finished, now starting transfers
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Waiting for transfers to finish
2023/08/27 03:02:56 DEBUG : Waiting for deletions to finish
2023/08/27 03:02:56 INFO  : There was nothing to transfer
2023/08/27 03:02:56 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:              3243 / 3243, 100%
Elapsed time:         0.3s

2023/08/27 03:02:56 DEBUG : 13 go routines active
2023/08/27 03:02:56 DEBUG : sftp://david@frier-nas-03.home.arpa:22/Backups/davidbkups/: Closing 1 unused connections

And when I run the same script from a cron job (also as david)

2023/08/27 02:59:54 DEBUG : rclone: Version "v1.63.1" starting with parameters ["rclone" "sync" "-vv" "/media/david/Seag8/davidbkups/" "QNap3:Backups/davidbkups/"]
2023/08/27 02:59:54 DEBUG : Creating backend with remote "/media/david/Seag8/davidbkups/"
2023/08/27 02:59:54 DEBUG : Using config file from "/home/david/.config/rclone/rclone.conf"
2023/08/27 02:59:54 DEBUG : fs cache: renaming cache item "/media/david/Seag8/davidbkups/" to be canonical "/media/david/Seag8/davidbkups"
2023/08/27 02:59:54 DEBUG : Creating backend with remote "QNap3:Backups/davidbkups/"
2023/08/27 02:59:54 Failed to create file system for "QNap3:Backups/davidbkups/": couldn't connect to ssh-agent: SSH agent requested but SSH_AUTH_SOCK not-specified
2

I think this bug is fixed already. Try the latest beta.

3

I think that it is not the same as rclone with --sftp-ssh option still checks for SSH_AUTH_SOCK · Issue #7235 · rclone/rclone · GitHub

Here we have rclone.config explicitly requesting to use ssh-agent:

key_use_agent = true

Its failure in crontab has nothing to do with rclone but with the way how crontab works.

@kahomono - either do not use key_use_agent or set SSH_AUTH_SOCK in crontab environment.

1 Like
4

Setting SSH_AUTH_SOCK worked, tho it feels like kind of a cheat.

Thanks!

5

run:

echo $SSH_AUTH_SOCK

in command line and you will see that this variable is set. But it is not in crontab environment.

It is very common problem people face using crontab.

1 Like
6

it seems to me i have many other rclone commands running from cron - but maybe not using ssh? I'll have to comb through them and see.

7

Ah yes, well spotted.

The usual solution is not to set SSH_AUTH_SOCK as the ssh agent may not be available when cron is running, but to use ssh identity files. These need to have no password, so should be locked down in other ways (eg by IP address or by command to run or both).

I guess if you are running cron off a user's crontab and you are always logged in and have given the key to the ssh agent then it will work.

8

so... if I am indeed using an identity file with no password, I can set key_use_agent = false ?

9

even you should:) or just remove it - default is false

--sftp-key-file

Path to PEM-encoded private key file.

Leave blank or set key-use-agent to use ssh-agent.

Leading ~ will be expanded in the file name as will environment variables such as ${RCLONE_CONFIG_DIR}.

Properties:

Config: key_file
Env Var: RCLONE_SFTP_KEY_FILE
Type: string
Required: false

you either provide key file or use key agent to source it for you

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.