Issues Working with --file-names

Help and Support
21

2020/06/15 13:22:23 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone.exe" "--config" "rclone.conf" "ls" "2019:2019\VIOL_IMAGES\LONG_TERM" "--files-from" "ls-test.txt" "--log-level" "DEBUG" "--max-depth" "1" "--log-file" "ls.singlefile.ff.txt"]
2020/06/15 13:22:23 DEBUG : Using config file from "C:\rclone\rclone.conf"
2020/06/15 13:23:06 DEBUG : 201810: Excluded
2020/06/15 13:23:06 DEBUG : 201901: Excluded
2020/06/15 13:23:06 DEBUG : 201902: Excluded
2020/06/15 13:23:06 DEBUG : 201903: Excluded
2020/06/15 13:23:06 DEBUG : 201904: Excluded
2020/06/15 13:23:06 DEBUG : 201905: Excluded
2020/06/15 13:23:06 DEBUG : 201906: Excluded
2020/06/15 13:23:06 DEBUG : 201907: Excluded
2020/06/15 13:23:06 DEBUG : 201908: Excluded
2020/06/15 13:23:06 DEBUG : 201909: Excluded
2020/06/15 13:23:06 DEBUG : 201910: Excluded
2020/06/15 13:23:06 DEBUG : 201911: Excluded
2020/06/15 13:23:06 DEBUG : 201912: Excluded
2020/06/15 13:23:06 DEBUG : 202001: Excluded
2020/06/15 13:23:06 DEBUG : 5 go routines active
2020/06/15 13:23:06 DEBUG : rclone: Version "v1.51.0" finishing with parameters ["rclone.exe" "--config" "rclone.conf" "ls" "2019:2019\VIOL_IMAGES\LONG_TERM" "--files-from" "ls-test.txt" "--log-level" "DEBUG" "--max-depth" "1" "--log-file" "ls.singlefile.ff.txt"]

ls-test.txt contents:
\201901\20190108\030707_\030707_201901081651090033258_0.jpg

22

I'm including this only to show the results when I "reclone copy" the same file:
2020/06/15 13:52:41 DEBUG : rclone: Version "v1.51.0" starting with parameters ["rclone.exe" "--config" "rclone.conf" "copy" "2019:2019\VIOL_IMAGES\LONG_TERM\201901\20190108\030707_\030707_201901081651090033258_0.jpg" "D:\SWIFTSTACK\TEST" "--log-level" "DEBUG" "--log-file" "cp.singlefile.txt"]
2020/06/15 13:52:41 DEBUG : Using config file from "C:\rclone\rclone.conf"
2020/06/15 13:52:41 DEBUG : 030707_201901081651090033258_0.jpg: Need to transfer - File not found at Destination
2020/06/15 13:52:41 DEBUG : 030707_201901081651090033258_0.jpg: MD5 = 499881b39327285147aead8acd432676 OK
2020/06/15 13:52:41 INFO : 030707_201901081651090033258_0.jpg: Copied (new)
2020/06/15 13:52:41 INFO :
Transferred: 2.791k / 2.791 kBytes, 100%, 65.477 kBytes/s, ETA 0s
Transferred: 1 / 1, 100%
Elapsed time: 0.0s

23

So what does:

rclone ls "2019:2019\VIOL_IMAGES\LONG_TERM201901\20190108\030707_\030707_201901081651090033258_0.jpg

show.

24

2858 30707_201901081651090033258_0.jpg

25

So when you run:

rclone ls "2019:2019\VIOL_IMAGES --files-from" "ls-test.txt"

is it just showing the one file?

26

nothing returned

27

Sorry as I typo'ed your path in a paste.

rclone ls "2019:2019\VIOL_IMAGES\LONG_TERM" --files-from" "ls-test.txt"
28

yea i caught that and corrected it.

i just tried what you had sent and trimmed it to 2019:2019\ to see if it would error and still the same - nothing returned

29

i didn't think it would make a difference but i noticed you put quote marks around the base path and the file name. tried that and no difference.

30

i also intentionally mis-keyed the file name and it immediately came back and said it could't find the file so I guess it is finding the file correctly.

31

does it make a difference if the file is created in a Windows environment vs. a Linux environment? i.e. CRLF vs CR?

32

The contents of the file should not matter.

I'm just a bit confused as if this worked:

rclone ls "2019:2019\VIOL_IMAGES\LONG_TERM201901\20190108\030707_\030707_201901081651090033258_0.jpg

this should work

rclone ls 2019:2019\VIOL_IMAGES\LONG_TERM201901 --files-from test.txt

with test.txt

20190108\030707_\030707_201901081651090033258_0.jpg
33

I think I've figure out the issue. Even though the tool will take the backslashes from the command line, you have to convert to all forward slashes when you use the --file-from option. I have had success converting all of the slashes and getting it to run and use the --file-from file. I'm still dealing with it being extremely slow but will work through that trying the different parameters.

Thanks for all your help today.

34

Hmm, that doesn't feel right as the issue though.

I wonder if @ncw or @calisro might be able to chime in ? I'm definitely not a Windows guy but I would think if you are on Windows, that should not be the case.

Maybe @asdffdsa could check also? You have me curious now!

35

I agree. The server that I'm running rclone from is Windows. The backend is SwiftStack on Linux. Converting to the forward slash did solve the issue but it's not working as fast I would have hoped. I have a rather large number of individual files (MM) that I need to download and was hoping this would be an efficient way of getting them. I actually do better by reading the file name from a text file in one script, and the calling another script to run the rclone copy, while the other is keeping track of the number of instances I've called that runs the rclone (i.e. imitating the --transfers 100 parm).

I don't want to diverge to the next issue on this thread but would welcome any suggestions.

36

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.