Teamdrive problem: ls, lsd... not working / when uploading everything gets duplicated

What is the problem you are having with rclone?

When using rclone with TeamDrive I can upload, but rclones does not check if the file exists, so everytime I upload stuff it gets duplicated, I end up having many files with exactly the same name, size.. etc.

I also noticed that when I run rclone ls teamdrive: it shows nothing, but if I go and check on the website there are all the files.

That only happens on teamdrives, on regular account or gsuite I can see and upload the normal way.

I've configured rclone in a headless machine, the way that I have to enter a URL and then give back a code.

What is your rclone version (output from rclone version)

1.50

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu Server 16.'4

Which cloud storage system are you using? (eg Google Drive)

Google Drive, Team Drive shared folder

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy /file teamdrive:/ -v -P
rclone ls teamdrive:

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

rclone: Version "v1.50.1" starting with parameters ["rclone" "-vv" "ls" "teamdrive:"]
2019/11/07 15:28:39 DEBUG : Using config file from "/home/USER/.config/rclone/rclone.conf"
2019/11/07 15:28:39 DEBUG : 5 go routines active
2019/11/07 15:28:39 DEBUG : rclone: Version "v1.50.1" finishing with parameters ["rclone" "-vv" "ls" "teamdrive:"]

You need to share the full log and the debug will show why it's re-uploading it.

Thanks for the asnswer, here is the full -vv log of two copyes for the same file to the same teamdrive, you can see that rclone says that there is no duplicate, when in fact, there is (which leads to the problem that it can not list the files on the teamdrive). I have to say that the problem happens on all diferents teamdrives, no only that one in particular.

Here is the log:

2019/11/07 16:05:46 DEBUG : rclone: Version "v1.50.1" starting with parameters ["rclone" "copy" "Backup_03-11-19.tgz" "teamdrive:" "-vv" "-P"]
2019/11/07 16:05:46 DEBUG : Using config file from "/home/USER/.config/rclone/rclone.conf"
2019-11-07 16:05:46 DEBUG : Backup_03-11-19.tgz: Need to transfer - File not found at Destination
2019-11-07 16:05:47 DEBUG : Backup_03-11-19.tgz: Sending chunk 0 length 8388608
2019-11-07 16:05:47 DEBUG : Backup_03-11-19.tgz: Sending chunk 8388608 length 8388608
2019-11-07 16:05:48 DEBUG : Backup_03-11-19.tgz: Sending chunk 16777216 length 8388608
2019-11-07 16:05:48 DEBUG : Backup_03-11-19.tgz: Sending chunk 25165824 length 8388608
2019-11-07 16:05:48 DEBUG : Backup_03-11-19.tgz: Sending chunk 33554432 length 8388608
2019-11-07 16:05:49 DEBUG : Backup_03-11-19.tgz: Sending chunk 41943040 length 8388608
2019-11-07 16:05:49 DEBUG : Backup_03-11-19.tgz: Sending chunk 50331648 length 8388608
2019-11-07 16:05:49 DEBUG : Backup_03-11-19.tgz: Sending chunk 58720256 length 8388608
2019-11-07 16:05:50 DEBUG : Backup_03-11-19.tgz: Sending chunk 67108864 length 8388608
2019-11-07 16:05:50 DEBUG : Backup_03-11-19.tgz: Sending chunk 75497472 length 8388608
2019-11-07 16:05:50 DEBUG : Backup_03-11-19.tgz: Sending chunk 83886080 length 8388608
2019-11-07 16:05:50 DEBUG : Backup_03-11-19.tgz: Sending chunk 92274688 length 8388608
2019-11-07 16:05:51 DEBUG : Backup_03-11-19.tgz: Sending chunk 100663296 length 8388608
2019-11-07 16:05:51 DEBUG : Backup_03-11-19.tgz: Sending chunk 109051904 length 8388608
2019-11-07 16:05:51 DEBUG : Backup_03-11-19.tgz: Sending chunk 117440512 length 8388608
2019-11-07 16:05:52 DEBUG : Backup_03-11-19.tgz: Sending chunk 125829120 length 8388608
2019-11-07 16:05:52 DEBUG : Backup_03-11-19.tgz: Sending chunk 134217728 length 8388608
2019-11-07 16:05:52 DEBUG : Backup_03-11-19.tgz: Sending chunk 142606336 length 8388608
2019-11-07 16:05:53 DEBUG : Backup_03-11-19.tgz: Sending chunk 150994944 length 7518673
2019-11-07 16:05:54 DEBUG : Backup_03-11-19.tgz: MD5 = 5c4a5e72f23674d77e5269aa667b8bdf OK
2019-11-07 16:05:54 INFO : Backup_03-11-19.tgz: Copied (new)
Transferred: 151.170M / 151.170 MBytes, 100%, 18.862 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 8s
2019/11/07 16:05:54 INFO :
Transferred: 151.170M / 151.170 MBytes, 100%, 18.862 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 8s

2019/11/07 16:05:54 DEBUG : 5 go routines active
2019/11/07 16:05:54 DEBUG : rclone: Version "v1.50.1" finishing with parameters ["rclone" "copy" "Backup_03-11-19.tgz" "teamdrive:" "-vv" "-P"]
Terminal:~$ rclone copy Backup_03-11-19.tgz teamdrive: -vv -P
2019/11/07 16:05:58 DEBUG : rclone: Version "v1.50.1" starting with parameters ["rclone" "copy" "Backup_03-11-19.tgz" "teamdrive:" "-vv" "-P"]
2019/11/07 16:05:58 DEBUG : Using config file from "/home/USER/.config/rclone/rclone.conf"
2019-11-07 16:05:59 DEBUG : Backup_03-11-19.tgz: Need to transfer - File not found at Destination
2019-11-07 16:05:59 DEBUG : Backup_03-11-19.tgz: Sending chunk 0 length 8388608
2019-11-07 16:05:59 DEBUG : Backup_03-11-19.tgz: Sending chunk 8388608 length 8388608
2019-11-07 16:06:00 DEBUG : Backup_03-11-19.tgz: Sending chunk 16777216 length 8388608
2019-11-07 16:06:00 DEBUG : Backup_03-11-19.tgz: Sending chunk 25165824 length 8388608
2019-11-07 16:06:00 DEBUG : Backup_03-11-19.tgz: Sending chunk 33554432 length 8388608
2019-11-07 16:06:00 DEBUG : Backup_03-11-19.tgz: Sending chunk 41943040 length 8388608
2019-11-07 16:06:01 DEBUG : Backup_03-11-19.tgz: Sending chunk 50331648 length 8388608
2019-11-07 16:06:01 DEBUG : Backup_03-11-19.tgz: Sending chunk 58720256 length 8388608
2019-11-07 16:06:01 DEBUG : Backup_03-11-19.tgz: Sending chunk 67108864 length 8388608
2019-11-07 16:06:01 DEBUG : Backup_03-11-19.tgz: Sending chunk 75497472 length 8388608
2019-11-07 16:06:02 DEBUG : Backup_03-11-19.tgz: Sending chunk 83886080 length 8388608
2019-11-07 16:06:02 DEBUG : Backup_03-11-19.tgz: Sending chunk 92274688 length 8388608
2019-11-07 16:06:02 DEBUG : Backup_03-11-19.tgz: Sending chunk 100663296 length 8388608
2019-11-07 16:06:02 DEBUG : Backup_03-11-19.tgz: Sending chunk 109051904 length 8388608
2019-11-07 16:06:03 DEBUG : Backup_03-11-19.tgz: Sending chunk 117440512 length 8388608
2019-11-07 16:06:03 DEBUG : Backup_03-11-19.tgz: Sending chunk 125829120 length 8388608
2019-11-07 16:06:03 DEBUG : Backup_03-11-19.tgz: Sending chunk 134217728 length 8388608
2019-11-07 16:06:03 DEBUG : Backup_03-11-19.tgz: Sending chunk 142606336 length 8388608
2019-11-07 16:06:04 DEBUG : Backup_03-11-19.tgz: Sending chunk 150994944 length 7518673
2019-11-07 16:06:05 DEBUG : Backup_03-11-19.tgz: MD5 = 5c4a5e72f23674d77e5269aa667b8bdf OK
2019-11-07 16:06:05 INFO : Backup_03-11-19.tgz: Copied (new)
Transferred: 151.170M / 151.170 MBytes, 100%, 24.498 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 6.1s
2019/11/07 16:06:05 INFO :
Transferred: 151.170M / 151.170 MBytes, 100%, 24.498 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 6.1s

2019/11/07 16:06:05 DEBUG : 5 go routines active
2019/11/07 16:06:05 DEBUG : rclone: Version "v1.50.1" finishing with parameters ["rclone" "copy" "Backup_03-11-19.tgz" "teamdrive:" "-vv" "-P"]

looks like file isn't on the destination.

What's your rclone.conf without keys/passwords?

Can you run a rclone ls before the copy and see if it's there?

This looks like the team drive inconsistent directory listing problem...

Try --disable ListR in the call to rclone ls

Team drives seem to take some time for the directory listings to catch up so if you do two copies quickly, you'll get duplicates, whereas if you wait some time between them you wont.

Well I can see the file on the webpage, and if I keep uploading it goes like 10 files with exactly the same namme on the same folder :S

Here is my rclone config

[teamdrive]
type = drive
scope = drive
shared_with_me = true
token = {"access_token":"XXXX.XXXXXXXXXX_XXXXXXXXXXXXXXXXXXXXjqvzKHV903qlTfeqRRqsaqStByJJO_02MXXXXXXXXXXI1KsyySg1_8xJXXXXXXXXXXLbbWPTn-4z_o4jOpXXXXXXXXXXWlcL98VU","token_type":"Bearer","refresh_token":"1//XXXXj$
team_drive = XXXXXXXXXDwW-UkXXXA

I run rclone ls after uploading a bunch of things, I can see and access them on the webpage but rclones sees nothing...

Same result using --disable ListR, it's true that when I upload filed I takes a bit of time to access them on Drive (website), but right know all the files I've uploaded are visible on the webpage and rclone ls still says there's nothing (I have things uploaded from several days ago).
There must be a problem somewhere... but I don't know how to proceed or what else to check

I hope the files will appear eventually via rclone. How long is it taking do you think?

It's been a few days since I uploaded a bunch of things, and the "rclone ls" stills shows nothing... while on the web there is everything available

I just noticed this in your config...

Try removing that - this should fix the duplicated uploads and most likely the rclone ls problems.

You are completely right! Now it's working like a charm.

Can I ask, what is the "shared_with_me" used for? How can I understand the use of that?

Many thanks

This is usually used by its command line flag --shared-with-me to allow you to see the files that people share with you which rclone doesn't normally show you. However the flag is exclusive - if you set it you only see shared with me stuff which was the problem here.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.