Error when rclone on Windows E:

Suspected Bug
1

I had a problem, but I don't know if it's regression!

What is the problem you are having with rclone?

With the V1.52

What is your rclone version (output from rclone version)

v1.54.0-beta.5119.20578f3f8

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Centos 7

Which cloud storage system are you using? (eg Google Drive)

sftp

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy -Pvvv DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/ /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/ --include * --no-update-modtime

A log from the command with the -vv flag

With the version 1.52, everything it's OK
But with the 1.54_Beta :
[jenkins@PIDSA-INGE1-DATA ~]$ rclone copy -Pvvv DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/ /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/ --include * --no-update-modtime

2021/02/01 14:47:40 DEBUG : rclone: Version "v1.54.0-beta.5119.20578f3f8" starting with parameters ["rclone" "copy" "-Pvvv" "DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/" "/data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/" "--include" "*" "--no-update-modtime"]

2021/02/01 14:47:40 DEBUG : Using config file from "/home/jenkins/.config/rclone/rclone.conf"

2021/02/01 14:47:40 DEBUG : Creating backend with remote "DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/"

2021/02/01 14:47:41 DEBUG : sftp://jenkins@10.99.20.1:22/E/applis/: New connection 10.99.30.1:55478->10.99.20.1:22 to "SSH-2.0-OpenSSH_for_Windows_8.1"

2021/02/01 14:47:41 DEBUG : sftp://jenkins@10.99.20.1:22/E/applis/: Using absolute root directory "/C:/Users/jenkins/E/applis"

2021/02/01 14:47:41 DEBUG : Creating backend with remote "/data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/"

2021-02-01 14:47:41 ERROR : : error reading source directory: directory not found

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for checks to finish

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for transfers to finish

2021-02-01 14:47:41 ERROR : Attempt 1/3 failed with 1 errors and: directory not found

2021-02-01 14:47:41 ERROR : : error reading source directory: directory not found

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for checks to finish

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for transfers to finish

2021-02-01 14:47:41 ERROR : Attempt 2/3 failed with 1 errors and: directory not found

2021-02-01 14:47:41 ERROR : : error reading source directory: directory not found

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for checks to finish

2021-02-01 14:47:41 DEBUG : Local file system at /data/DOSSIER_PARTAGE-INGE1_TM-PEMPS_tmp/: Waiting for transfers to finish

2021-02-01 14:47:41 ERROR : Attempt 3/3 failed with 1 errors and: directory not found

2

hello,

  • the two commands that were posted, they have different sources?
  • not sure that it matters, but this should not be needed --include *
  • would be helpful to see that config file
  • try testing with rclone lsd on the root.
3
  • the two commands that were posted, they have different sources?
    => it's the same but I installed V1,52 after
  • not sure that it matters, but this should not be needed --include *
    => yes but the same result
  • would be helpful to see that config file
    => yes I post it
4

the sources are different between the two commands?
DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/
and
DOSSIER_PARTAGE-INGE1_TM-PEMPS:/applis/

5

OK excuse me, but it's this command :
DOSSIER_PARTAGE-INGE1_TM-PEMPS:E/applis/ for V1,52 et V1,54beta

6

what is the output of
lsd DOSSIER_PARTAGE-INGE1_TM-PEMPS: -vv
lsd DOSSIER_PARTAGE-INGE1_TM-PEMPS:/E/ -vv

7

I modifie my post

8

did not see the config file?

9

[DOSSIER_PARTAGE-INGE1_TM-PEMPS]

type = sftp

host = 10.99.20.1

user = jenkins

port = 22

pass = vbwvWy2jU8q_VnUNON-fhhQAwmGuYHF5rQ

md5sum_command = md5sum

sha1sum_command = sha1sum

10

what is the output of
rclone lsd DOSSIER_PARTAGE-INGE1_TM-PEMPS: --recursive -vv

11

with witch version of rclone?

12

when posting config files, redact id and passwords.

test with the latest stable, v1.53.4

13

I test just now with 1,54...
rclone copy -Pv /var/lib/jenkins/MAJ_SEP/20210205-008-core3sdsv5i64.exe CEMG-TM:C:/temp/SEP.exe 2021-02-15 12:51:08 ERROR : : error reading source directory: directory not found
2021-02-15 12:51:08 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2021-02-15 12:51:08 ERROR : : error reading source directory: directory not found
2021-02-15 12:51:08 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2021-02-15 12:51:08 ERROR : : error reading source directory: directory not found
2021-02-15 12:51:08 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.5s
2021/02/15 12:51:08 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.5s

2021/02/15 12:51:08 Failed to copy: directory not found

For me there is a problem...

If I do what you asked to me :
[jenkins@PIDSA-DTG-JENKINS root]$ rclone lsd CEMG-TM: --recursive -vv
2021/02/15 12:54:46 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "lsd" "CEMG-TM:" "--recursive" "-vv"]
2021/02/15 12:54:46 DEBUG : Using config file from "/var/lib/jenkins/.config/rclone/rclone.conf"
2021/02/15 12:54:46 DEBUG : Creating backend with remote "CEMG-TM:"
2021/02/15 12:54:46 DEBUG : sftp://jenkins@10.24.20.1:22/: New connection 10.240.100.1:58408->10.24.20.1:22 to "SSH-2.0-OpenSSH_for_Windows_8.1"
2021/02/15 12:54:46 DEBUG : sftp://jenkins@10.24.20.1:22/: Using absolute root directory "/C:/Users/jenkins"
-1 2020-06-22 14:48:41 -1 AppData
-1 2020-06-22 14:48:41 -1 Application Data
-1 2020-06-22 14:48:41 -1 Cookies
-1 2016-07-16 15:23:21 -1 Desktop
-1 2020-06-22 14:48:41 -1 Documents
-1 2016-07-16 15:23:21 -1 Downloads
-1 2016-07-16 15:23:21 -1 Favorites
-1 2016-07-16 15:23:21 -1 Links
-1 2020-06-22 14:48:41 -1 Local Settings
-1 2016-07-16 15:23:21 -1 Music
-1 2020-06-22 14:48:41 -1 My Documents
-1 2020-06-22 14:48:41 -1 NetHood
-1 2016-07-16 15:23:21 -1 Pictures
-1 2020-06-22 14:48:41 -1 PrintHood
-1 2020-06-22 14:48:41 -1 Recent
-1 2016-07-16 15:23:21 -1 Saved Games
-1 2020-06-22 14:48:41 -1 SendTo
-1 2020-06-22 14:48:41 -1 Start Menu
-1 2020-06-22 14:48:41 -1 Templates
-1 2016-07-16 15:23:21 -1 Videos
-1 2020-06-22 14:48:41 -1 Local Settings/Application Data
-1 2020-06-22 14:48:41 -1 Local Settings/History
-1 2016-07-16 15:23:21 -1 Local Settings/Microsoft
-1 2021-02-10 07:03:16 -1 Local Settings/Temp
-1 2020-06-22 14:48:41 -1 Local Settings/Temporary Internet Files
-1 2020-06-22 14:48:41 -1 My Documents/My Music
-1 2020-06-22 14:48:41 -1 My Documents/My Pictures
-1 2020-06-22 14:48:41 -1 My Documents/My Videos
-1 2021-01-26 18:15:56 -1 Recent/AutomaticDestinations
-1 2021-01-26 18:15:56 -1 Recent/CustomDestinations
I can't see the E:

14

rclone thinks that var/lib/jenkins/MAJ_SEP/20210205-008-core3sdsv5i64.exe is a folder, not a file.

if you want to rename a file, then use https://rclone.org/commands/rclone_copyto/
"This can be used to upload single files to other than their current name"

15

My problem come with other windows DISK C: no pb, with E: I got a pb.

This command rclone copy -Pv /var/lib/jenkins/MAJ_SEP/20210205-008-core3sdsv5i64.exe CEMG-TM:C:/temp/SEP.exe, work with rclone 1,52

16

work with rclone 1,52
run that exact same command using 1.52 and post the debug log

rclone thinks this is a folder, not a file; thus the error error reading source directory: directory not found
/var/lib/jenkins/MAJ_SEP/20210205-008-core3sdsv5i64.exe

you are trying to copy and rename a file, is that correct?
if true, then try rclone copyto

17

So :
[jenkins@PIDSA-DTG-JENKINS root]$ rclone move -Pvvv ENV-TM-EXT:E:/fusion_inventory /var/lib/jenkins/fusion_inventory
2021/02/15 16:13:00 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "move" "-Pvvv" "ENV-TM-EXT:E:/fusion_inventory" "/var/lib/jenkins/fusion_inventory"]
2021/02/15 16:13:00 DEBUG : Using config file from "/var/lib/jenkins/.config/rclone/rclone.conf"
2021/02/15 16:13:00 DEBUG : Creating backend with remote "ENV-TM-EXT:E:/fusion_inventory"
2021/02/15 16:13:00 DEBUG : sftp://jenkins@10.61.20.2:22/E:/fusion_inventory: New connection
2021/02/15 16:13:00 DEBUG : sftp://jenkins@10.61.20.2:22/E:/fusion_inventory: Using absolute root directory "/C:/Users/jenkins/E:/fusion_inventory"
2021/02/15 16:13:00 DEBUG : Creating backend with remote "/var/lib/jenkins/fusion_inventory"
2021-02-15 16:13:00 ERROR : : error reading source directory: directory not found
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for checks to finish
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for transfers to finish
2021-02-15 16:13:00 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2021-02-15 16:13:00 ERROR : : error reading source directory: directory not found
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for checks to finish
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for transfers to finish
2021-02-15 16:13:00 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2021-02-15 16:13:00 ERROR : : error reading source directory: directory not found
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for checks to finish
2021-02-15 16:13:00 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for transfers to finish
2021-02-15 16:13:00 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.4s
2021/02/15 16:13:00 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors: 1 (retrying may help)
Elapsed time: 0.4s

2021/02/15 16:13:00 DEBUG : 11 go routines active
2021/02/15 16:13:00 Failed to move: directory not found

And with 1,52 :
[jenkins@PIDSA-DTG-JENKINS root]$ rclone move -Pvvv ENV-TM-EXT:E:/fusion_inventory /var/lib/jenkins/fusion_inventory
2021/02/15 16:13:28 DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone" "move" "-Pvvv" "ENV-TM-EXT:E:/fusion_inventory" "/var/lib/jenkins/fusion_inventory"]
2021/02/15 16:13:28 DEBUG : Using config file from "/var/lib/jenkins/.config/rclone/rclone.conf"
2021/02/15 16:13:29 DEBUG : sftp://jenkins@10.61.20.2:22/E:/fusion_inventory: New connection
2021-02-15 16:13:29 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for checks to finish
2021-02-15 16:13:29 DEBUG : Local file system at /var/lib/jenkins/fusion_inventory: Waiting for transfers to finish
2021-02-15 16:13:29 INFO : There was nothing to transfer
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time: 0.0s
2021/02/15 16:13:29 INFO :
Transferred: 0 / 0 Bytes, -, 0 Bytes/s, ETA -
Elapsed time: 0.0s

2021/02/15 16:13:29 DEBUG : 11 go routines active

18

I see this line : 2021/02/15 16:13:00 DEBUG : sftp://jenkins@10.61.20.2:22/E:/fusion_inventory: Using absolute root directory "/C:/Users/jenkins/E:/fusion_inventory"
With version after 1,52, it's seems to add a new parameter

I find this : from the change log
SFTP

  • Use the absolute path instead of the relative path for listing for improved compatibility (Nick Craig-Wood)
19

Use the absolute path instead of the relative path for listing for improved compatibility
pretty sure, that is just for readability in the logs files, nothing to do with the command itself.

as for the command, https://rclone.org/sftp/
"Paths are specified as remote:path. If the path does not begin with a / it is relative to the home directory of the user. An empty path remote: refers to the user's home directory.

20

[jenkins@PIDSA-DTG-JENKINS root]$ rclone move -Pvvv ENV-TM-EXT:/E:/fusion_inventory /var/lib/jenkins/fusion_inventory
Works
THX