I already have the data I want to upload on the cloud. But I have changed the folder structure on my local drive. The folder is not synced to the cloud. Now I want my google drive to mirror the local folder structure without having to upload all the files again. I just want the drive to rearrange the files into the new folder structure.
Can rclone do this? Or any other software? Is it possible to use a file comparison software on onedrive placeholders to rearrange them? Thank you.
First post, if you need additional info, please comment.
When you sync you can use --track-renames and rclone will automatically do this for you.
It will compare checksums on the local files and see if a file with that hash exists already (somewhere else or with a different name) on the server, and just server-side move /rename them instead of re-copying.
Note that this requires that compatible checksums are possible between the two remotes. This means:
It won't work between 2 different cloud remotes that use different hashes
It won't work between an unencrypted remote and an encrypted one (because the hashes won't match)
Your local PC probably doesn't have a filesystem with hashes, but it can make any of them them on the fly (and will do so if needed for --track-renames to work).
You may be able to work around the unencrypted local --> encrypted remote by simply storing the stuff locally encrypted with a encrypt + "local" type remote. As long as you use the same encryption key then the hashes will match and --track-renames will be possible.
Me and Nick have had some discussions on how to work around some of the limitations because this feature is just so useful it would be nice to have it work in more use-cases.
Finally - you may want to try this first using the --dry-run flag. This only simulates what would happen so you can test a command without making a mistake and potentially deleting something by accident.
Hope this is useful and if you need more help then you should share your spesific setup (ie. do you use an encrypted remote ect.)
Thank you so much for your reply. I will test it and report back.
And thank you for your hard work on this software. It is dead useful with just the sync command. I don't have to reupload everything.
I just finished trying it out. It worked. I was confused at first because it doesn't create empty folders, but I figured it out after a minute. You are a life saver. Thank you.
Thank you. I really appreciate it.
Another question, how do I move the files that are to be moved to recycle bin since they are not in the source folder to a new folder instead of the recycle bin?
--backup-folder will make it so that any time a file would been deleted or overwritten - it will be moved to the backup folder instead. The backup folder will keep 1 revision of old files in case you need to recover something.
It sounds like that may be what you want.
Also keep in mind that Google drive by default uses it's own trash. Anything deleted is actually moved into a temporary 30-day trash before they are deleted. They can be recovered within this time.
I guess nothing is stopping you from using both if you want to have extra retention for safety though.
Are --backup-folder and --backup-dir= the same?
Also, is it possible to move files in the google drive trash folder to a new folder inside the drive? It doesn't support viewing the files in the trash if they are in a folder and restoring them messes up the sync.
Thank you for all your help.
--backup-dir is the correct command, and this is what I meant when I said folder.
If you want to view your google-trash then use:
--drive-trashed-only
This shows only trashed files. They still appear to be in the same place they were deleted from, so don't be confused by that. All files there should be normally accessible I'm pretty sure, regardless of being inside folders or not. If you want to restore something you can just move the files from the trash-remote to the Gdrive remote like you would move any other file.
You should set up a separate remote/mount for this trash-view and try your best to not confuse it, so choose a good name ect.
So I tried it and it works. I did exactly what you told me not not, got confused by the folders which were not trashed, then I realized that the ones in the main directory were actually empty and only the trashed ones had any files in them. Thank you so much.
Still getting used to mount though. I tried to move all the files not in a folder to a new folder and I got the --vfs-cache-mode error thing. Which one should I use to ensure that copy and moving files works fine ?
Also, can the cache directory have files in it or does it need to be empty ?
Caching isn't strictly needed for most simple reads and writes (most of the errors are not critical) - but it can't do every action that an OS would expect a drive to handle like that. Especially any operations that involve editing or opening for write+read are pretty limited without caching. For full compatibility with all operations you will need
--vfs-cache-mode writes
That does have some small downsides in the sense that all writes first get written to the local disk before being uploaded - but that's a fairly small price to pay. If you upload a lot every day you might want to put the VFS cache directory on a HDD rather than an SSD to avoid unnecessary wear (although that is becoming less and less of an issue on modern SSDs).
If you did your transfers in the command-line then there is no need to write-cache as you don't have to deal with OS compatibility - so if you need to move large amounts of files in bulk then keep that in mind. If it's not a lot of data then using the mount is fine for the convenience of it.
I would definitely use an empty directory for the cache to avoid any confusion, but I don't think it is strictly required.
One last thing. Usually moving around files on a drive can be done by just telling the server to move the file (rather than downloading it and re-uploading it). This is obviously a lot faster and more efficient and it can be done on a mount. However - rclone might not be able to recognize that it can do this when the trash and the drive are separate mounts, and I don't think there is a good way to put both together in the same mount.
So in short: If there are many large files - you may want to do that in the command-line as you can leverage server-side moves and avoid any bandwidth limitations.
Maybe someone else knows of a good way to work around this on mounts. I just don't access to trash very often outside of emergency recovery so I haven't looked into it much.
