`lsf ` adding tabs in output

Suspected Bug
1

What is the problem you are having with rclone?

When I run lsf with --format hipst --hash MD5 on a Google Drive I am getting unexpected \t (tabs) in the output after the id field, as well as extra 'column'.

E.g. (xxx inserted for obfuscation, and spaces around the \t for clarity):

2b796c0b2ad9324afe4a5592f13de6ca;1_Hj9kaxxx3umJ11Uy4bnuq27xY0hJVt9 \t 16TyD5ECIxxxjfORgM_6bVaRSol2nmaQ;Cocina-HuevosRancheros-422HQ.mov;1325346660;2018-03-14 05:19:53
c0db06721dcdb7c235fff97ab289e61a;1ar_xLCxxxwxfLZHzdM7RFhBIpAvrzYFW \t 1hOUEfJS7xxxBzkchndBZfRSz0r4Bevo;Cocina-QuinoaHerbSalad-v3.mp4;75151935;2018-11-05 05:43:27
371536540f5da2d6267d79942f6c97af;1UKg6SWxxxD8iAoiGzco90J4VgEiF2UuS \t 1oBfQhRfJxxxmGtBzApCs_oQRCzhcVJU;Cocina-LemonPoppyPancakes-square-v2-flipped.mp4;73131493;2018-09-14 00:51:40

Most of the other millions of files look like this without the tab and extra id (?):

f567fd11fa36eb97c7943a3796670716;1kBmnC5xxxG43yj9h4RGLniOoKoDoC96J;Cocina-LEMON CAKE V3z.mpeg;305891532;2020-08-22 10:01:53
b2faff48f3a3fff090ede390a84439ac;1IAJZFhxxxuaK7KeSg6i2-F4YQcDOm9EI;Cocina-MojitoSmoothie-v2-4444-flat.mov;1877760868;2019-10-24 12:05:39
ef7c3c6c590e22fa644e042678dac847;1VKUFLfxxxDBWn_Nr5UvJkASexL8d53Pq;Cocina-MojitoSmoothie-4444_16bit.mov;1885263044;2019-10-24 12:05:39

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 6.5.0-1023-oracle (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.22.1
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

d="SHELLY/COCINA/FINAL HQ EXPORTS"; rclone lsf gd0:"$d" --files-only --fast-list --checkers 200 --format hipst --hash MD5 -R | tee test.txt

A log from the command with the -vv flag

2024/05/22 00:06:13 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "-vv" "lsf" "gd0:SHELLY/COCINA/FINAL HQ EXPORTS" "--files-only" "--fast-list" "--checkers" "200" "--format" "hipst" "--hash" "MD5" "-R" "--log-file" "log.txt"]
2024/05/22 00:06:13 DEBUG : Creating backend with remote "gd0:SHELLY/COCINA/FINAL HQ EXPORTS"
2024/05/22 00:06:13 DEBUG : Using config file from "/home/ubuntu/.config/rclone/rclone.conf"
2024/05/22 00:06:13 DEBUG : Google drive root 'SHELLY/COCINA/FINAL HQ EXPORTS': 'root_folder_id = 0xxxU07aR9pFUk9PVA' - save this in the config to speed up startup
2024/05/22 00:06:26 DEBUG : 6 go routines active
2

hello,
i was not able to reproduce on windows and linux.

rclone lsf gdrive01:test --files-only --fast-list --checkers 200 --format hipst --hash MD5 -R -vv | tee test.txt
2024/05/22 10:43:59 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "lsf" "gdrive01:test" "--files-only" "--fast-list" "--checkers" "200" "--format" "hipst" "--hash" "MD5" "-R" "-vv"]
2024/05/22 10:43:59 DEBUG : Creating backend with remote "gdrive01:test"
2024/05/22 10:43:59 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/05/22 10:43:59 DEBUG : Google drive root 'test': 'root_folder_id = 0AIYnsu88uXytUk9PVA' - save this in the config to speed up startup
2024/05/22 10:44:00 DEBUG : 6 go routines active
c4ca4238a0b923820dcc509a6f75849b;1JgSNPqB519vFKKyOu_K8-pxF7pQzz-B2;file.ext;1;2023-01-08 13:19:00
44d88612fea8a8f36de82e1278abb02f;1UyxljUQgqPVHi_A8RXnHf3-l0HA1dxAw;eicar.com;68;2023-01-08 13:19:00

and is screenshot of test.txt from nano with display of whitespace enabled

image
image1210×75 6.15 KB

3

I dug deeper and it seems that at least for some of these, there are actually 2 ids being reported, separated by a tab, where the first one is the actual file, and the second is Mime: application/vnd.google-apps.shortcut

So rclone is reporting both original and shortcut ... maybe this should be documented.

I would like a way to NOT report the shortcut.

4

Aha! So I need to use --drive-skip-shortcuts (see Google drive)

Solved

6

Try with a folder that contains shortcuts.

7

sorry, confused, as you already marked this topic as solved??

8

Yes, just explaining how to reproduce.

9

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.