Sync only makes makes my local files & folders disappear

What is the problem you are having with rclone?

Sync only makes makes my local files & folders completely disappear.

What is your rclone version (output from rclone version)

rclone v1.53.2

  • os/arch: windows/amd64
  • go version: go1.15.2

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10, 64 bit

Which cloud storage system are you using? (eg Google Drive)

OneDrive (personal)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --progress RCLOnedrive: OneDrive
Transferred:       65.141M / 65.141 MBytes, 100%, 314.242 kBytes/s, ETA 0s
Checks:           2308366 / 2308366, 100%
Deleted:            63289
Transferred:            9 / 9, 100%
Elapsed time:     30m11.8s

The rclone config contents with secrets removed.

type = onedrive
token = {"access_token":"EwCA[...REDACTED...]rCz1a3Jfj4A$","expiry":"2020-11-10T12:54:25.9767821-05:00"}
drive_id = ca[...REDACTED...]d64
drive_type = personal

A log from the command with the -vv flag

Note: My original command ("rclone sync --progress RCLOnedrive: OneDrive") did not include -vv.
I ran it again later with rclone sync --progress --update --dry-run --log-file=rclonelog.txt -vv RCLOnedrive: OneDrive
The rclone log is 540mb, so I don't think I can paste it here.
Here's the beginning part of the log

2020/11/10 14:54:06 DEBUG : rclone: Version "v1.53.2" starting with parameters ["rclone" "sync" "--progress" "--update" "--dry-run" "--log-file=rclonelog.txt" "-vv" "RCLOnedrive:" "OneDrive"]
2020/11/10 14:54:06 DEBUG : Using config file from "C:\\Users\\Name\\.config\\rclone\\rclone.conf"
2020/11/10 14:54:06 DEBUG : Creating backend with remote "RCLOnedrive:"
2020/11/10 14:54:07 DEBUG : Creating backend with remote "OneDrive"
2020/11/10 14:54:07 DEBUG : fs cache: renaming cache item "OneDrive" to be canonical "//?/D:/Cloud/rclone/OneDrive"
2020/11/10 14:54:07 NOTICE: git-log.txt: Skipped copy as --dry-run is set
2020/11/10 14:54:07 NOTICE: randomfile.ipynb: Skipped copy as --dry-run is set

After those logs, it's pretty much hundreds of thousands of lines of:

2020/11/10 14:54:07 DEBUG : randomfolder/ Destination mod time is within 1s of source and files identical, skipping

The log finally ends with:

2020/11/10 15:22:45 DEBUG : Local file system at //?/D:/Cloud/rclone/OneDrive: Waiting for transfers to finish
2020/11/10 15:22:45 DEBUG : Waiting for deletions to finish
2020/11/10 15:22:45 NOTICE: Jupyter_Notebooks: Skipped remove directory as --dry-run is set
2020/11/10 15:22:45 DEBUG : Local file system at //?/D:/Cloud/rclone/OneDrive: deleted 1 directories
2020/11/10 15:22:45 INFO  : 
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:           2245081 / 2245081, 100%
Transferred:            2 / 2, 100%
Elapsed time:     28m38.6s

2020/11/10 15:22:45 DEBUG : 17 go routines active

So I was foolish for 2 reasons:

  1. I used the same sync command 2 weeks ago, and my local files & folders were gone. Not in recycling bin, not anywhere. I used "Everything" for windows to confirm. Indeed, the files & folders simply vanished. Yet, despite all this, I convinced myself these were fake memories and that I never had the files & folders in the local rclone OneDrive folder. Now that it's happened again, I know this is real.

  2. I know I should have done a dry run first, but with millions of files, I thought it'd take days. Turns out, it takes 30 minutes.

Anyways, from reading the forums and docs, I thought Rclone sync had zero chance of deleting local files. It's purely a 1-way push to the cloud. So I'm incredibly confused why this is happening.

Note: Though my folder is named OneDrive, it is NOT the actual OneDrive folder (that syncs with the OneDrive app).

Can anyone help me?

hello and welcome to the forum,

that is not correct,
rclone does a one-way sync from source to destination and can delete files in the destination.
these are possible syncs

  • local to cloud
  • cloud to local
  • cloud to cloud
  • local to local

in your case, it is cloud to local, so you can expect rclone to delete local files.
the cloud is RCLOnedrive: and the local is OneDrive

as per the log

  • renaming cache item "OneDrive" to be canonical "//?/D:/Cloud/rclone/OneDrive"
  • Local file system at //?/D:/Cloud/rclone/OneDrive: deleted

Wow really appreciate the quick response. Its good to know it's my own screwup.

If I'm reading correctly, this can be fixed by switching the two, correct?
So it should be:

rclone sync --progress OneDrive: RCLOnedrive

Additionally, to be clear, per how I unintentionally used the sync feature, are my files permanently gone?
EDIT: Deleted my longwinded context. Yea, the files & folders are absolutely gone. Unless someone has good news, and I'm misunderstanding.

So it should be:
what are you trying to do, local to cloud?

as a side note, wiztree is much faster and accurate than windirstat and treesize.

Yes, local to cloud. Is the command correct?

rclone sync --progress OneDrive: RCLOnedrive

And thanks for the recommendation. Will look into it right now.

sorry, that is not correct but you are getting closer.

  • a remote MUST have a colon in the name.
  • i would use full path for local folders.

rclone sync --progress D:\Cloud\rclone\OneDrive RCLOnedrive:

1 Like

Awesome, thanks for this. Seriously, you're great.

Does anyone know if the files & folders are permanently deleted (or a way to recover)? Maybe a temporary bin I don't know?
Going to leave this topic open (for a few days) just in case anybody knows, before I mark your answer as Solution.

there is no temp bin.

you mentioned that there are files in the recycle bin, so why not recover them?
there is a recycle bin icon on the desktop, have you double-click that?

you can try

you might be able to do a system restore.

None of the files or folders in my Recycling Bin are the ones in the OneDrive folder unfortunately. No luck with Recuva either.

All good, it'll take maybe a day to redo all the files I lost. Many thanks once again.

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.