Rename duplicates on-the-go without renaming the files on Drive

Help and Support
1

What is the problem you are having with rclone?

See, my Drive has duplicates, which is causing issues with my copy operation. I was trying to rename them using rclone dedupe, but as most of them are Google Classroom files, I don't own them. In that sense, I get an error trying to rename most of them. I want to run dedupe while the copy operation is happening, but only on local.

What is your rclone version (output from rclone version)

rclone v1.56.0
- os/version: debian 10.9 (64 bit)
- os/kernel: 5.10.0-0.bpo.5-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.5
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy drive: .

The rclone config contents with secrets removed.

[drive]
type = drive
scope = drive
root_folder_id =
token = {"access_token":"SECRET","token_type":"Bearer","refresh_token":"SECRET","expiry":"2021-08-31T13:37:01.347743968-04:00"}
team_drive =

A log from the command with the -vv flag

2021/08/31 13:22:24 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "copy" "-vv" "drive:" "."]
2021/08/31 13:22:24 DEBUG : Creating backend with remote "drive:"
2021/08/31 13:22:24 DEBUG : Using config file from "/home/thetechrobo/.config/rclone/rclone.conf"
2021/08/31 13:22:24 DEBUG : Google drive root '': 'root_folder_id = 0ANICP9YV5YpcUk9PVA' - save this in the config to speed up startup
2021/08/31 13:22:24 DEBUG : Creating backend with remote "."
2021/08/31 13:22:24 DEBUG : fs cache: renaming cache item "." to be canonical "/home/thetechrobo/rclone/secret"
[snip]
2021/08/31 13:22:30 NOTICE: Classroom/Immersion Francais 4／5/Les Regularites dans les nombres et la geometrie.pptx: Duplicate object found in source - ignoring
[snip]
2

hello and welcome to the forum,

you want to dedupe on local?
not sure what that means, as local filesystem cannot have two files with the same name.

can you explain in more detail your use-case?

1 Like
3

Hey, thanks for the quick answer!

I have duplicates in my Drive and I'm worried that NONE are getting through.

I see the log messages for those duplicate files, but I don't see the folder "Classroom" on my local machine. That makes me worry that the duplicates aren't getting through AT ALL. Hence, me wanting to deduplicate files as they're downloaded.

Is that correct, or is that simply because rclone doesn't respect directory structure or something?

4

about that folder,

  • are all the names in there duplicates
  • are there any files that are not duplicates?

without a debug log, hard to know what exactly is going on..

  • add this to your command, --log-level=DEBUG --log-file=rclone.log
    *run the command
  • post the log.
5

yes, most of them.

try refreshing the page, i added a log after I noticed i didn't fill it out. i snipped to the main parts of the log that are required..

but here it is:

2021/08/31 13:22:24 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "copy" "-vv" "drive:" "."]
2021/08/31 13:22:24 DEBUG : Creating backend with remote "drive:"
2021/08/31 13:22:24 DEBUG : Using config file from "/home/thetechrobo/.config/rclone/rclone.conf"
2021/08/31 13:22:24 DEBUG : Google drive root '': 'root_folder_id = 0ANICP9YV5YpcUk9PVA' - save this in the config to speed up startup
2021/08/31 13:22:24 DEBUG : Creating backend with remote "."
2021/08/31 13:22:24 DEBUG : fs cache: renaming cache item "." to be canonical "/home/thetechrobo/rclone/secret
[snip]
2021/08/31 13:22:30 NOTICE: Classroom/Immersion Francais 4／5/Les Regularites dans les nombres et la geometrie.pptx: Duplicate object found in source - ignoring
[snip]

After running rclone for a while, I see that folder populated, but not with that file.

or does it do that duplicate check BEFORE it downloads any files?

i'm a bit lost here.

6

Okay, figured it out.

It seems to do the dupe check BEFORE it downloads the stuff.

I now see that file inside the folder, about a half hour of running rclone.

Thanks for your help!

8

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.