Refresh-times with server-side-across-configs?

Help and Support
1

Hi, I am syncing from a crypt on Drive to a crypt on Dropbox where the only change is files is modtime (but modtime was changed on all files).

When trying to use --refresh-times to minimize unnecessary data transfer, I noticed the error in the debug log src and dst identical but can't set mod time without deleting and re-uploading so tried adding --server-side-across-configs to --refresh-times. (My hope was that the former flag could allow Dropbox to do a server-side copy, if necessary, just to update the modtime). Unfortunately, this didn't change anything and my otherwise unmodified files are all being flagged for transfer.

Is there currently a combo of features/flags that might enable rclone to simply change the modtime on my crypt-on-Dropbox remote without re-uploading (when it's a sync target from a crypt on Drive source)?

2

For Dropbox, you can't modify the time without re-uploading so that doesn't work to my knowledge via server side either as it's just moving it.

You don't want to move it as you want to update the time with refresh and that can only be done with a new upload.

You can see it in the debug:

rclone touch DB:hosts -vv
2023/04/04 08:54:52 DEBUG : Setting --config "/opt/rclone/rclone.conf" from environment variable RCLONE_CONFIG="/opt/rclone/rclone.conf"
2023/04/04 08:54:52 DEBUG : rclone: Version "v1.62.2" starting with parameters ["rclone" "touch" "DB:hosts" "-vv"]
2023/04/04 08:54:52 DEBUG : Creating backend with remote "DB:"
2023/04/04 08:54:52 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2023/04/04 08:54:52 DEBUG : Touch time 2023-04-04 08:54:52.202015156 -0400 EDT m=+0.120063810
2023/04/04 08:54:52 DEBUG : DB: Loaded invalid token from config file - ignoring
2023/04/04 08:54:52 DEBUG : Saving config "token" in section "DB" of the config file
2023/04/04 08:54:52 DEBUG : Keeping previous permissions for config file: -rwxrwxr-x
2023/04/04 08:54:52 DEBUG : DB: Saved new token in config file
2023/04/04 08:54:52 DEBUG : Dropbox root '': Touching "hosts"
2023/04/04 08:54:52 ERROR : Attempt 1/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/04/04 08:54:52 DEBUG : Touch time 2023-04-04 08:54:52.528631154 -0400 EDT m=+0.446679835
2023/04/04 08:54:52 DEBUG : Dropbox root '': Touching "hosts"
2023/04/04 08:54:52 ERROR : Attempt 2/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/04/04 08:54:52 DEBUG : Touch time 2023-04-04 08:54:52.623922088 -0400 EDT m=+0.541970811
2023/04/04 08:54:52 DEBUG : Dropbox root '': Touching "hosts"
2023/04/04 08:54:52 ERROR : Attempt 3/3 failed with 1 errors and: failed to touch: can't set modified time without deleting existing object
2023/04/04 08:54:52 DEBUG : 6 go routines active
2023/04/04 08:54:52 INFO  : Dropbox root '': Committing uploads - please wait...
2023/04/04 08:54:52 Failed to touch: failed to touch: can't set modified time without deleting existing object
3

Ah, thank you.

UPDATED

From what I can tell of the Dropbox V2 API, there is an indication that it is possible to update mtime via files_move_v2; however, presumably this requires a move to a temporary location, and move back to the original/desired location. Might this be worth supporting?

4

Feel free to submit a PR if you think it's possible as we'd be happy to accept it.

5

Alas, I lack the skill to do the implementation; however, I would be more than happy to contribute if I can be useful in other respects: creating an issue, testing, writing documentation.

6

Sure - feel free to raise an enhancement request on GitHub and you can tag this forum post.

7

I looked at the docs and I don't see that. Can you explain further?

If we can support setting a modtime on an existing object in dropbox it would make it a lot more usable.

8

I apologize for the noise and what I gather may have raised everyone's hopes alongside my own. I didn't run this down sufficiently: cloud - Is there a way to UPDATE file metadata through dropbox api with Python? - Stack Overflow

9

If you do find a way please let us know. Dropbox keep adding new bits to their APIs so I wouldn't be suprised to find it in there somewhere.

Here is the latest I found in the dropbox developer forum about this:

10

Thank you. I certainly will.

For good measure (and posterity), I asked via Dropbox Developer Support ticket and received the following (re-)confirmation:

H-----, Apr 4, 2023, 7:20 AM PDT:

Hi,

Thanks for writing in. No, unfortunately the Dropbox API doesn't offer a way to update a file's 'client_modified' without uploading the file, but I'll pass this along as a feature request. I can't promise if or when that might be implemented though.

Regards,
H-----

Not being familiar with the Dropbox API, I also asked whether a custom file_properties element might be an avenue for storing a timestamp as a mutable alternative to client_modified (as well as any anticipated trade-offs) and it was explained that file_properties are scoped to an app-user so do not seem a great alternative for rclone:

H-----, Apr 4, 2023, 8:19 AM PDT:

Yes, the Dropbox API offers "file properties" functionality which allows third party apps to set and get custom metadata. These file property values are only accessible to the specific app-user, so if you only need this information for your own app and only accessible to the connected user, you can use this as an alternative to relying on the standard 'client_modified'. Getting and setting file properties does not require re-uploading a file.

I would expect some minor performance cost as it would involve more server work to get/set file properties in addition to the standard file data and metadata operations when performing calls with the file properties included.

Also, file properties are completely under the control of the third party app and don't correlate to or sync with the standard file metadata fields such as 'client_modified', so custom file property and standard file metadata values would not be automatically consistent and you'd need to ensure your app uses one or the other in its logic as appropriate.

Apologies again for the noise! I'll certainly post any relevant updates I come across.

Thank you!

1 Like
11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.