So I just moved 450GB of data from Amazon Drive to Backblaze B2 using a server with gigabit speeds. I did it with a server because my home internet had 5mbps upload, and uploading 450GB of data would take at least 2 weeks of 24/7 uploading.
I just tried to do a copy from my home server to Backblaze (which already has all the data because I transferred it from Amazon Drive) just to make sure everything was working. rclone decided that it wanted to reupload every single file because of some mod time questions that it had.
Is there any way I can just tell it to update the mod time for all the files that have been uploaded? They are the same files on my side and Backblaze’s side, but something happened when the files were moved from Amazon Drive to Backblaze that made rclone think that the modification times had changed. I was able to use --no-update-modtime but that isn’t exactly a fix. It just prevents the modification time from being updated (and also prevents the entire file from being reuploaded).
Is the only way to fix the mod times to reupload everything? I did the math and it’ll take 14 days straight, but if that’s what it takes to fix the issue then I don’t really mind waiting. I’ve got a second set of the data on Amazon Drive still and the server is set up with RAID 5 so I’m not really concerned with losing the data at this point. Just want to have something online that will successfully work long-term (and allow me to use mod time).
I ran into this problem transferring from ACD to GDrive. I know GDrive lets us modify the times but rclone doesn’t have that functionality. Check out this thread where we modified a bit of it to do the update… That being said, I am not sure if backblaze allows update of mod times via their API.
“Modified times are used in syncing and are fully supported except in the case of updating a modification time on an existing object. In this case the object will be uploaded again as B2 doesn’t have an API method to set the modification time independent of doing an upload.”
You’re only option is to do a ‘–size-only’ or a ‘–checksum’ depending on your use-case. Or re-upload your data. Alternatively you could sync the local files to have the same modtime as b2. Depends on your needs. There is a ‘find’ command in the link above that helps with that if needed.
Syncing the local files to have the same modtime seems like it could be the better/faster thing to do. Modtimes aren’t really important for these files (they’re just picture backups). How would I go about syncing the local files to have the same modtime? It looks like you’re doing the reverse of what I want to do with GDrive (updating the GDrive modtimes). Thanks.
Capture the output and then you can run it. It will be a whole lot of touch commands so you can verify it before you actually make changes. Just change the /path/dir/ and the remote:
Keep in mind that some remotes don’t hold dates as accurately so you will like need to add something like this for a margin of error to your future syncs with b2: --modify-window 1ms
Thanks. I’ll give that command a try then see what rclone thinks afterwards. What’s the “/path/dir” for exactly though? It’s a little confusing seeing all that in one big chunk.
The path to where the mapping is to the remote. Run it as is and you will see what I mean… It won’t change anything as it just outputs the touch commands… It doesn’t actually run them…
It seems to have worked with most of the files, but some were still having issues for whatever reason. I’m not sure why though because I checked and it looked like all the files that were going to be reuploaded were definitely in the list of touch commands and the dates looked correct in those commands. It might be easier to just reupload everything. Thanks for the help either way.
Yeah I did that as well. The things that were off were off by something like 20 hours usually. It was either a time difference of 0 or a time difference of multiple hours.
