Rclone lists directories as files

sapuizait · April 9, 2021, 8:54am

What is the problem you are having with rclone?

starting from today rclone list some of my directories as files - directories are still there but the typical '''rclone ls''' will not show me the directory! Instead it prints the name of the directory as a file!

What is your rclone version (output from `rclone version`)

rclone v1.53.3

os/arch: linux/amd64
go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu 18.04 64bit

Which cloud storage system are you using? (eg Google Drive)

amazon s3

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

Paste '''rclone ls s3-fecogenome:fecogenome-data/MAGs_before_dRep''' here

The rclone config contents with secrets removed.

Paste config here

A log from the command with the `-vv` flag

Paste  2021/04/09 10:55:44 DEBUG : rclone: Version "v1.53.3" starting with parameters ["rclone" "ls" "-vv" "s3-fecogenome:fecogenome-data/MAGs_before_dRep"]
2021/04/09 10:55:44 DEBUG : Using config file from "/home/panos/.rclone.conf"
2021/04/09 10:55:44 DEBUG : Creating backend with remote "s3-fecogenome:fecogenome-data/MAGs_before_dRep"
2021/04/09 10:55:45 DEBUG : fs cache: adding new entry for parent of "s3-fecogenome:fecogenome-data/MAGs_before_dRep", "s3-fecogenome:fecogenome-data"
2021/04/09 10:55:45 DEBUG : Hungate: Excluded
2021/04/09 10:55:45 DEBUG : MAGs_before_dRep: Excluded
2021/04/09 10:55:45 DEBUG : N10_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N10_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N11_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N11_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N12_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N12_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N13_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N13_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N14_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N14_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N15_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N15_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N16_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N16_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N17_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N17_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N18_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N18_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N19_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N19_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N1_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N1_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N20_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N20_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N21_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N21_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N22_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N22_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N23_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N23_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N24_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N24_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N25_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N25_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N26_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N26_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N27_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N27_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N28_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N28_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N29_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N29_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N2_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N2_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N30_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N30_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N31_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N31_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N32_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N32_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N33_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N33_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N34_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N34_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N35_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N35_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N36_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N36_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N37_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N37_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N38_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N38_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N39_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N39_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N3_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N3_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N40_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N40_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N41_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N41_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N42_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N42_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N43_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N43_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N44_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N44_not_host_autometa_new: Excluded
2021/04/09 10:55:45 DEBUG : N44_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N45_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N45_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N46_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N46_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N47_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N47_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N48_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N48_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N4_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N4_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N5_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N5_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N6_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N6_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N7_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N7_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N8_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N8_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : N9_not_host_autometa: Excluded
2021/04/09 10:55:45 DEBUG : N9_not_host_megahit: Excluded
2021/04/09 10:55:45 DEBUG : R_analysis: Excluded
2021/04/09 10:55:45 DEBUG : Ruben: Excluded
2021/04/09 10:55:45 DEBUG : WINTER_data: Excluded
2021/04/09 10:55:45 DEBUG : _autometa: Excluded
2021/04/09 10:55:45 DEBUG : emapper: Excluded
2021/04/09 10:55:45 DEBUG : kma: Excluded
2021/04/09 10:55:45 DEBUG : kma_IMG_VR_DB: Excluded
2021/04/09 10:55:45 DEBUG : merged.master.files: Excluded
2021/04/09 10:55:45 DEBUG : merged.orfs: Excluded
2021/04/09 10:55:45 DEBUG : metadata: Excluded
2021/04/09 10:55:45 DEBUG : metaxa: Excluded
2021/04/09 10:55:45 DEBUG : N10_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N10_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N11_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N11_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N11_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N11_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N11_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N11_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N12_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N12_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N12_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N12_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N12_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N12_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N13_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N13_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N13_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N13_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N13_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N13_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N14_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N14_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N14_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N14_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N14_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N14_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N15_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N15_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N15_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N15_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N15_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N15_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N16_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N16_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N16_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N16_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N16_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N16_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N17_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N17_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N17_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N17_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N17_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N17_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N18_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N18_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N18_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N18_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N18_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N18_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N19_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N19_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N19_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N19_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N19_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N19_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N1_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N1_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N1_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N1_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N1_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N1_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N20_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N20_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N20_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N20_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N20_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N20_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N21_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N21_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N21_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N21_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N21_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N21_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N22_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N22_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N22_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N22_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N22_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N22_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N23_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N23_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N23_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N23_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N23_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N23_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N24_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N24_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N24_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N24_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N24_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N24_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N25_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N25_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N25_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N25_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N25_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N25_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N26_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N26_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N26_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N26_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N26_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N26_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N27_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N27_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N27_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N27_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N27_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N27_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N28_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N28_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N28_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N28_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N28_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N28_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N29_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N29_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N29_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N29_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N29_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N29_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N2_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N2_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N2_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N2_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N2_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N2_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N30_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N30_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N30_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N30_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N30_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N30_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N31_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N31_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N31_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N31_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N31_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N31_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N32_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N32_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N32_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N32_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N32_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N32_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N33_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N33_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N33_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N33_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N33_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N33_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N34_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N34_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N34_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N34_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N34_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N34_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N35_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N35_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N35_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N35_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N35_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N35_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N36_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N36_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N36_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N36_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N36_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N36_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N37_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N37_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N37_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N37_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N37_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N37_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N38_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N38_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N38_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N38_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N38_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N38_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N39_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N39_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N39_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N39_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N39_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N39_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N3_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N3_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N3_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N3_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N3_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N3_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N40_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N40_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N40_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N40_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N40_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N40_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N41_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N41_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N41_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N41_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N41_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N41_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N42_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N42_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N42_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N42_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N42_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N42_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N43_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N43_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N43_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N43_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N43_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N43_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N44_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N44_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N44_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N44_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N44_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N44_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N45_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N45_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N45_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N45_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N45_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N45_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N46_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N46_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N46_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N46_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N46_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N46_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N47_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N47_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N47_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N47_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N47_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N47_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N48_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N48_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N48_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N48_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N48_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N48_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N4_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N4_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N4_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N4_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N4_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N4_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N5_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N5_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N5_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N5_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N5_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N5_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N6_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N6_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N6_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N6_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N6_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N6_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N7_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N7_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N7_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N7_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N7_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N7_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N8_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N8_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N8_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N8_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N8_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N8_not_host.2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N9_L4_1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N9_L4_1.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N9_L4_2.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N9_L4_2.fq.gz_trimming_report.txt: Excluded
2021/04/09 10:55:45 DEBUG : N9_not_host.1.fq.gz: Excluded
2021/04/09 10:55:45 DEBUG : N9_not_host.2.fq.gz: Excluded
    12358 MAGs_before_dRep
2021/04/09 10:55:45 DEBUG : 4 go routines active
 here

sapuizait · April 9, 2021, 9:51am

when I try to list the entire cloud, this is what I get

MAGs_before_dRep
MAGs_before_dRep/N10_DBSCAN_round101_4.fas
MAGs_before_dRep/N10_DBSCAN_round101_4.txt
MAGs_before_dRep/N10_DBSCAN_round119_20.fas
MAGs_before_dRep/N10_DBSCAN_round119_20.txt
MAGs_before_dRep/N10_DBSCAN_round130_6.fas
...

clearly, the directory is there but there is also an object/file with identical name that is masking my directory
I suspect if I delete the object the problem will be solved but I do not want to risk deleting the directory

And most importantly, why did this suddenly happen?

ncw · April 9, 2021, 4:45pm

I think you are correct.

I'm not sure you'll be able to delete this file with rclone... rclone deletefile might do it but I suspect it will fail.

Did you use some different software? Some software (eg cyberduck) creates these 0 length directory markers.

sapuizait · April 9, 2021, 4:56pm

ok, rclone delete worked - i was a bit scared but i only tried it after I had copied all the important files to a temp directory

I simply did

rclone delete s3-fecogenome:fecogenome-data/MAGs_before_dRep

and the object/file is now gone while (thank god) the directory is still there

I am using rclone on my ubuntu pc (v1.53) and in a remote slurm cluster, the rclone version in the cluster is v1.37. It is an old version but still, this should not have happened... right?

I have also installed RCX on my phone, an android app that allows you to have rclone access from your phone but I have only used very few times

again, thanks

ncw · April 9, 2021, 5:03pm

Glad you got rid of the pesky file!

No it shouldn't.

It is possible that you could upload a file with the same name as a directory with rclone somehow.

That uses rclone under the hood so shouldn't be a problem.

sapuizait · April 9, 2021, 5:36pm

OK, thanks! will keep an eye out and hopefully this wont happen again

system · April 12, 2021, 5:37pm

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.