Optimization of Class A and Class B operations

cberni · June 25, 2024, 3:24am

We are incurring a very high cost for class A and B operations when performing move operations with rclone. We conduct this move operation only to verify if the file is already in the cloud and then delete it locally. These files have already been uploaded previously.

We use the move operation because, if for any reason the file is not in the bucket, it uploads the file before permanently deleting it.

As you can see in the attached graph, on June 11th, we performed the move operation to delete an entire month of data that was on the local machine. The folder for this month contains an average of 500GB and 2072617 files.

Below is the command used:

rclone move /LOCAL_PATH REMOTE:PATH --no-traverse --fast-list --delete-empty-src-dirs

rclone v1.65.0
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-167-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.4
- go/linking: static
- go/tags: none

google cloud storage

We suspect that rclone might be performing more checks than necessary. Could you help us debug this and see if we can find a way to optimize it?
I can move an entire month's worth of data again using the necessary commands for logging that you provide.

asdffdsa · June 25, 2024, 12:22pm

can debug it yourself, using --dump=headers
and post the output of rclone config redacted remote:

i know with aws deep glacier, there are api calls for the files itself, and additional api calls and storage for metadata.
aws has a pricing calculator, so i can know exactly what to expect...

cberni · June 25, 2024, 10:08pm

[remote]
type = google cloud storage
service_account_file = /etc/file.json

For testing, I selected a smaller folder with only 432 files. In total, there were twice as many checks.

Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:               864 / 864, 100%
Deleted:              432 (files), 240 (dirs)
Elapsed time:        13.5s

move.log (683.3 KB)

asdffdsa · June 25, 2024, 10:29pm

perhaps, rclone selfupdate, test on a single directory with a single file, without --delete-empty-src-dir
then post that dump log.

cberni · June 26, 2024, 1:57am

the same move with just one file...

2024/06/25 22:53:36 NOTICE: Automatically setting -vv as --dump is enabled
2024/06/25 22:53:36 DEBUG : rclone: Version "v1.65.0" starting with parameters ["rclone" "move" "/local/teste" "remote:test_bucket/teste" "--no-traverse" "--fast-list" "--delete-empty-src-dirs" "--config" "/etc/rclone.conf" "--dump=headers"]
2024/06/25 22:53:36 DEBUG : Creating backend with remote "/local/teste"
2024/06/25 22:53:36 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/25 22:53:36 DEBUG : Creating backend with remote "remote:test_bucket/teste"
2024/06/25 22:53:36 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:36 DEBUG : HTTP REQUEST (req 0xc000795000)
2024/06/25 22:53:36 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.65.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:36 DEBUG : HTTP RESPONSE (req 0xc000795000)
2024/06/25 22:53:36 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Wed, 26 Jun 2024 01:53:36 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/25 22:53:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:36 DEBUG : HTTP REQUEST (req 0xc000795300)
2024/06/25 22:53:36 DEBUG : GET /storage/v1/b/test_bucket/o/teste?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.65.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.21.4 gdcl/0.148.0
Accept-Encoding: gzip

2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:36 DEBUG : HTTP RESPONSE (req 0xc000795300)
2024/06/25 22:53:36 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Wed, 26 Jun 2024 01:53:36 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NqslUSC9THACmstOaMr5L2700DFZQ_Iv4Z-Sxbm-kbbWMbCPiV4-oKzEsPs4TVA_7JEeCdhroTQsg

2024/06/25 22:53:36 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:36 DEBUG : HTTP REQUEST (req 0xc000795800)
2024/06/25 22:53:36 DEBUG : GET /storage/v1/b/test_bucket/o/teste%2Fteste.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.65.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.21.4 gdcl/0.148.0
Accept-Encoding: gzip

2024/06/25 22:53:36 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/25 22:53:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:37 DEBUG : HTTP RESPONSE (req 0xc000795800)
2024/06/25 22:53:37 DEBUG : HTTP/2.0 200 OK
Content-Length: 801
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Wed, 26 Jun 2024 01:53:37 GMT
Etag: CJHXw7GU+IYDEAE=
Expires: Wed, 26 Jun 2024 01:53:37 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NollnSGNhUSf62Euq6zBDBd06N0YYdR0XeF8KhbR05DBTX6n15SjpbXJ69VRylPNj9YdjkAEyZF2Q

2024/06/25 22:53:37 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/25 22:53:37 DEBUG : GCS bucket test_bucket path teste: Waiting for checks to finish
2024/06/25 22:53:37 DEBUG : teste.txt: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/06/25 22:53:37 DEBUG : teste.txt: Unchanged skipping
2024/06/25 22:53:37 INFO  : teste.txt: Deleted
2024/06/25 22:53:37 DEBUG : GCS bucket test_bucket path teste: Waiting for transfers to finish
2024/06/25 22:53:37 INFO  : There was nothing to transfer
2024/06/25 22:53:37 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 2 / 2, 100%
Deleted:                1 (files), 0 (dirs)
Elapsed time:         1.1s

2024/06/25 22:53:37 DEBUG : 6 go routines active

cberni · June 27, 2024, 3:08am

selfupdate done.
--delete-empty-src-dir removed.

2024/06/27 00:02:58 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "move" "/local/test2" "remote:test_bucket/test2" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 00:02:58 DEBUG : Creating backend with remote "/local/test2"
2024/06/27 00:02:58 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 00:02:58 DEBUG : Creating backend with remote "remote:test_bucket/test2"
2024/06/27 00:02:59 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 00:02:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:02:59 DEBUG : HTTP REQUEST (req 0xc00066a480)
2024/06/27 00:02:59 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 00:02:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:03:02 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:02 DEBUG : HTTP RESPONSE (req 0xc00066a480)
2024/06/27 00:03:02 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 03:03:02 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 00:03:02 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:03:02 DEBUG : HTTP REQUEST (req 0xc000e8e6c0)
2024/06/27 00:03:02 DEBUG : GET /storage/v1/b/test_bucket/o/test2?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 00:03:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:03:02 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:02 DEBUG : HTTP RESPONSE (req 0xc000e8e6c0)
2024/06/27 00:03:02 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 03:03:02 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NpWcGTbkCJC6oN4p2p-EA4tCD7GmuzvwKKt-zVo2Ed2hGjpj3e8yNfLQtgUKNzO7dhnfAj30nTGnA

2024/06/27 00:03:02 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:03:02 DEBUG : HTTP REQUEST (req 0xc000936240)
2024/06/27 00:03:02 DEBUG : GET /storage/v1/b/test_bucket/o/test2%2Ftest2.txt?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 00:03:02 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 00:03:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:03 DEBUG : HTTP RESPONSE (req 0xc000936240)
2024/06/27 00:03:03 DEBUG : HTTP/2.0 200 OK
Content-Length: 801
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 03:03:03 GMT
Etag: CJPgxsrl+oYDEAE=
Expires: Thu, 27 Jun 2024 03:03:03 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NpA-j07UTtga7akzQV8lXtawtq1q_fq6ywNd6mTswCwbN6RBq3SoQOMHRwWV7PYoLU7L18y6m2UMQ

2024/06/27 00:03:03 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 00:03:03 DEBUG : GCS bucket test_bucket path test2: Waiting for checks to finish
2024/06/27 00:03:03 DEBUG : test2.txt: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/06/27 00:03:03 DEBUG : test2.txt: Unchanged skipping
2024/06/27 00:03:03 INFO  : test2.txt: Deleted
2024/06/27 00:03:03 DEBUG : GCS bucket test_bucket path test2: Waiting for transfers to finish
2024/06/27 00:03:03 INFO  : There was nothing to transfer
2024/06/27 00:03:03 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 2 / 2, 100%
Deleted:                1 (files), 0 (dirs), 29 B (freed)
Elapsed time:         8.7s

2024/06/27 00:03:03 DEBUG : 7 go routines active

asdffdsa · June 27, 2024, 12:48pm

looks ok to me, three api calls: one for token, two for checks.

i would try to script that, using something like

rclone check src: dst: --missing-on-dst=missing-on-dst.lst --fast-list --size-only
rclone move src: dst: --files-from=missing-on-dst.lst --no-traverse --no-check-dest

i do not use gcs, so not 100% the exact best set of flags to best reduce api calls.

cberni · June 27, 2024, 1:58pm

need to delete on /local after check.

cberni · June 27, 2024, 2:10pm

move if the file doesn't exist on the remote:

2024/06/27 10:53:09 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "move" "/local/test3" "remote:test_bucket/test3" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 10:53:09 DEBUG : Creating backend with remote "/local/test3"
2024/06/27 10:53:09 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 10:53:09 DEBUG : Creating backend with remote "remote:test_bucket/test3"
2024/06/27 10:53:09 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 10:53:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:09 DEBUG : HTTP REQUEST (req 0xc00072de60)
2024/06/27 10:53:09 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 10:53:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:09 DEBUG : HTTP RESPONSE (req 0xc00072de60)
2024/06/27 10:53:09 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:53:09 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 10:53:09 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:09 DEBUG : HTTP REQUEST (req 0xc000ad0c60)
2024/06/27 10:53:09 DEBUG : GET /storage/v1/b/test_bucket/o/test3?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:53:09 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:10 DEBUG : HTTP RESPONSE (req 0xc000ad0c60)
2024/06/27 10:53:10 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:53:10 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NrF-dVoaw2tRn3pizRhpsvslni3F4bxVHsPfWEaZ0O31lLmjapRkUxGRLjNyWJ9RWtWutg

2024/06/27 10:53:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:10 DEBUG : HTTP REQUEST (req 0xc0005a6120)
2024/06/27 10:53:10 DEBUG : GET /storage/v1/b/test_bucket/o/test3%2Ftest3?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:53:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:10 DEBUG : HTTP RESPONSE (req 0xc0005a6120)
2024/06/27 10:53:10 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 181
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:53:10 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NoNhz1smFC6Eiase84cbNd8mwmyxPZ_D53LnqwpsqRr7d1nxaUfdiLV8uaKMw5igIFPrzXP7nvmUw

2024/06/27 10:53:10 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:10 DEBUG : test3: Need to transfer - File not found at Destination
2024/06/27 10:53:10 DEBUG : GCS bucket test_bucket path test3: Waiting for checks to finish
2024/06/27 10:53:10 DEBUG : GCS bucket test_bucket path test3: Waiting for transfers to finish
2024/06/27 10:53:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:10 DEBUG : HTTP REQUEST (req 0xc0005a67e0)
2024/06/27 10:53:10 DEBUG : GET /storage/v1/b/test_bucket/o?alt=json&maxResults=1&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:53:10 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:11 DEBUG : HTTP RESPONSE (req 0xc0005a67e0)
2024/06/27 10:53:11 DEBUG : HTTP/2.0 200 OK
Content-Length: 1100
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:53:11 GMT
Expires: Thu, 27 Jun 2024 13:53:11 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0Nr0VloxY_8GeZyxWp_bDe6QYU09Wd2T1vj99S0QJY_G_lMxvtzaOq4pZrbGeDlx5DK9cNA_6GlCnw

2024/06/27 10:53:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:11 DEBUG : HTTP REQUEST (req 0xc000ad0360)
2024/06/27 10:53:11 DEBUG : POST /upload/storage/v1/b/test_bucket/o?alt=json&name=test3%2Ftest3&predefinedAcl=private&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=4816140e7ebec82e1356eeb58f3b8316553f1e977199751911e39f656ada
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:53:11 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:53:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:11 DEBUG : HTTP RESPONSE (req 0xc000ad0360)
2024/06/27 10:53:11 DEBUG : HTTP/2.0 200 OK
Content-Length: 784
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:53:11 GMT
Etag: CN2w9pz3+4YDEAE=
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NoczBbd0C5-SS2vX86AS2sniWZ922ecZaHtXoHOOHheHiuLrw4-xBowxCxTITwzVlC0znehyS_9YQ

2024/06/27 10:53:11 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:53:11 DEBUG : test3: md5 = 3044a983d76e7ac39871fcf4de14167e OK
2024/06/27 10:53:11 INFO  : test3: Copied (new)
2024/06/27 10:53:11 INFO  : test3: Deleted
2024/06/27 10:53:11 INFO  :
Transferred:             19 B / 19 B, 100%, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Deleted:                1 (files), 0 (dirs), 19 B (freed)
Renamed:                1
Transferred:            1 / 1, 100%
Elapsed time:         2.4s

2024/06/27 10:53:11 DEBUG : 7 go routines active

sync if the file doesn't exist on the remote:

2024/06/27 10:55:20 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "sync" "/local/test4" "remote:test_bucket/test4" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 10:55:20 DEBUG : Creating backend with remote "/local/test4"
2024/06/27 10:55:20 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 10:55:20 DEBUG : Creating backend with remote "remote:test_bucket/test4"
2024/06/27 10:55:20 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 10:55:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:20 DEBUG : HTTP REQUEST (req 0xc0007c8ea0)
2024/06/27 10:55:20 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 10:55:20 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:21 DEBUG : HTTP RESPONSE (req 0xc0007c8ea0)
2024/06/27 10:55:21 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:55:21 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 10:55:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:21 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:21 DEBUG : HTTP REQUEST (req 0xc0004c5680)
2024/06/27 10:55:21 DEBUG : GET /storage/v1/b/test_bucket/o/test4?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:55:21 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:21 DEBUG : HTTP RESPONSE (req 0xc0004c5680)
2024/06/27 10:55:21 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:55:21 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0Np2eK0Y2_BTLufH6ewGXl33LPOjdngJFTHK-AnwsSOGvndcdI9G5gde38yTEzA6yqJtqHUghvJetQ

2024/06/27 10:55:21 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:21 ERROR : Ignoring --no-traverse with sync
2024/06/27 10:55:21 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:21 DEBUG : HTTP REQUEST (req 0xc00050e5a0)
2024/06/27 10:55:21 DEBUG : GET /storage/v1/b/test_bucket/o?alt=json&maxResults=1000&prefix=test4%2F&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:55:21 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:22 DEBUG : HTTP RESPONSE (req 0xc00050e5a0)
2024/06/27 10:55:22 DEBUG : HTTP/2.0 200 OK
Content-Length: 26
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:55:21 GMT
Expires: Thu, 27 Jun 2024 13:55:21 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NpTieEwliVIvGtmZJtRX9ZeHTuJoPus7R00c1PMknia3cV2Qh3hlOkJR1urJUApvs1BVuTxprsYNg

2024/06/27 10:55:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:22 DEBUG : test4: Need to transfer - File not found at Destination
2024/06/27 10:55:22 DEBUG : GCS bucket test_bucket path test4: Waiting for checks to finish
2024/06/27 10:55:22 DEBUG : GCS bucket test_bucket path test4: Waiting for transfers to finish
2024/06/27 10:55:22 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:22 DEBUG : HTTP REQUEST (req 0xc00074f320)
2024/06/27 10:55:22 DEBUG : POST /upload/storage/v1/b/test_bucket/o?alt=json&name=test4%2Ftest4&predefinedAcl=private&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=08bd9329439e13d44e25bf3a0f83141f95ebf70a1468230c4d7245a9586d
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:55:22 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:55:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:22 DEBUG : HTTP RESPONSE (req 0xc00074f320)
2024/06/27 10:55:22 DEBUG : HTTP/2.0 200 OK
Content-Length: 784
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:55:22 GMT
Etag: CPnFltv3+4YDEAE=
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NqguLufMbeLvrDwAxDC6QMrnGkzfNhuDHmJT7h9hPBsrNCW9i5ktbiVPTgG8ovEbS0lSVaBt3bnOA

2024/06/27 10:55:22 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:55:22 DEBUG : test4: md5 = e2ad37cf109d152343361673bf839c23 OK
2024/06/27 10:55:22 INFO  : test4: Copied (new)
2024/06/27 10:55:22 DEBUG : Waiting for deletions to finish
2024/06/27 10:55:22 INFO  :
Transferred:             31 B / 31 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         1.4s

2024/06/27 10:55:22 DEBUG : 8 go routines active

copy if the file doesn't exist on the remote:

2024/06/27 10:56:04 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "/local/test5" "remote:test_bucket/test5" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 10:56:04 DEBUG : Creating backend with remote "/local/test5"
2024/06/27 10:56:04 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 10:56:04 DEBUG : Creating backend with remote "remote:test_bucket/test5"
2024/06/27 10:56:04 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 10:56:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:04 DEBUG : HTTP REQUEST (req 0xc000921440)
2024/06/27 10:56:04 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 10:56:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:04 DEBUG : HTTP RESPONSE (req 0xc000921440)
2024/06/27 10:56:04 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:56:04 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 10:56:04 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:04 DEBUG : HTTP REQUEST (req 0xc000920120)
2024/06/27 10:56:04 DEBUG : GET /storage/v1/b/test_bucket/o/test5?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:56:04 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:05 DEBUG : HTTP RESPONSE (req 0xc000920120)
2024/06/27 10:56:05 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:56:05 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NrvD193sKM-7T9QPY-olGPzQ3jSaGPW9cvI9EDkCEK3fiFUg5Gi72-RQRMXrydCdo7DFpD2_AGqLQ

2024/06/27 10:56:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:05 DEBUG : HTTP REQUEST (req 0xc000596b40)
2024/06/27 10:56:05 DEBUG : GET /storage/v1/b/test_bucket/o/test5%2Ftest5?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:56:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:05 DEBUG : HTTP RESPONSE (req 0xc000596b40)
2024/06/27 10:56:05 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 181
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:56:05 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NoEZ103T0SYA0j3-IPH1Q_DwMPA6kJ0wNtkj_bakxp1aTdjmQ3HP627nLdJFIAZ27PxBK9i1mWzwA

2024/06/27 10:56:05 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:05 DEBUG : test5: Need to transfer - File not found at Destination
2024/06/27 10:56:05 DEBUG : GCS bucket test_bucket path test5: Waiting for checks to finish
2024/06/27 10:56:05 DEBUG : GCS bucket test_bucket path test5: Waiting for transfers to finish
2024/06/27 10:56:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:05 DEBUG : HTTP REQUEST (req 0xc000921200)
2024/06/27 10:56:05 DEBUG : GET /storage/v1/b/test_bucket/o?alt=json&maxResults=1&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:56:05 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:06 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:06 DEBUG : HTTP RESPONSE (req 0xc000921200)
2024/06/27 10:56:06 DEBUG : HTTP/2.0 200 OK
Content-Length: 1100
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:56:05 GMT
Expires: Thu, 27 Jun 2024 13:56:05 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0Nqp3ZmWXgDhTh8DGAuQ3fPv4wlsIJaiHAHuf0QOF8miel0JmTRTDHUSNzfppf8olWPCASgb1ZphPQ

2024/06/27 10:56:06 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:06 DEBUG : HTTP REQUEST (req 0xc0008de6c0)
2024/06/27 10:56:06 DEBUG : POST /upload/storage/v1/b/test_bucket/o?alt=json&name=test5%2Ftest5&predefinedAcl=private&prettyPrint=false&uploadType=multipart HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Transfer-Encoding: chunked
Authorization: XXXX
Content-Type: multipart/related; boundary=9e34831ad3a2e687be48716d97b77c92b2c4874a7714aebfda3e54249bd4
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 10:56:06 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 10:56:06 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:06 DEBUG : HTTP RESPONSE (req 0xc0008de6c0)
2024/06/27 10:56:06 DEBUG : HTTP/2.0 200 OK
Content-Length: 784
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 13:56:06 GMT
Etag: CL+RqPD3+4YDEAE=
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NowKJ-DLkstkBCJyW54tIizeFKbiAU-h3h-1VusPlGn6wKw5oAGmsogCPtZP6nst7L8-Iz8oYDnVQ

2024/06/27 10:56:06 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 10:56:06 DEBUG : test5: md5 = 582bd5cb3873c5b34daced84c5e4fab4 OK
2024/06/27 10:56:06 INFO  : test5: Copied (new)
2024/06/27 10:56:06 INFO  :
Transferred:             35 B / 35 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         1.9s

2024/06/27 10:56:06 DEBUG : 8 go routines active

cberni · June 27, 2024, 2:18pm

move if the file exists on the remote:

2024/06/27 11:15:26 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "move" "/local/test3" "remote:test_bucket/test3" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 11:15:26 DEBUG : Creating backend with remote "/local/test3"
2024/06/27 11:15:26 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 11:15:26 DEBUG : Creating backend with remote "remote:test_bucket/test3"
2024/06/27 11:15:26 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 11:15:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:26 DEBUG : HTTP REQUEST (req 0xc000b95320)
2024/06/27 11:15:26 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 11:15:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:26 DEBUG : HTTP RESPONSE (req 0xc000b95320)
2024/06/27 11:15:26 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:15:26 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 11:15:26 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:26 DEBUG : HTTP REQUEST (req 0xc0007b67e0)
2024/06/27 11:15:26 DEBUG : GET /storage/v1/b/test_bucket/o/test3?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:15:26 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:27 DEBUG : HTTP RESPONSE (req 0xc0007b67e0)
2024/06/27 11:15:27 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:15:27 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NrtUtFjOqPh5J4Dwz7VwlRv6mnmY5vYSSC5NxixAOjgID9ezSsewAsurvtsVD0FqSQ_otwn1R-G0A

2024/06/27 11:15:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:27 DEBUG : HTTP REQUEST (req 0xc000a866c0)
2024/06/27 11:15:27 DEBUG : GET /storage/v1/b/test_bucket/o/test3%2Ftest3?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:15:27 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:27 DEBUG : HTTP RESPONSE (req 0xc000a866c0)
2024/06/27 11:15:27 DEBUG : HTTP/2.0 200 OK
Content-Length: 784
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:15:27 GMT
Etag: CN2w9pz3+4YDEAE=
Expires: Thu, 27 Jun 2024 14:15:27 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0Nr3IwNsj4iKJ380datc8qRUYDJm6LagTexDmQKdK3HS5ADGxRSindG7q1dzPjfp3gbjL_7QG9NDSg

2024/06/27 11:15:27 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:27 DEBUG : GCS bucket test_bucket path test3: Waiting for checks to finish
2024/06/27 11:15:27 DEBUG : test3: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/06/27 11:15:27 DEBUG : test3: Unchanged skipping
2024/06/27 11:15:27 INFO  : test3: Deleted
2024/06/27 11:15:27 DEBUG : GCS bucket test_bucket path test3: Waiting for transfers to finish
2024/06/27 11:15:27 INFO  : There was nothing to transfer
2024/06/27 11:15:27 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 2 / 2, 100%
Deleted:                1 (files), 0 (dirs), 19 B (freed)
Elapsed time:         1.3s

2024/06/27 11:15:27 DEBUG : 6 go routines active

sync if the file exists on the remote:

2024/06/27 11:15:59 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "sync" "/local/test4" "remote:test_bucket/test4" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 11:15:59 DEBUG : Creating backend with remote "/local/test4"
2024/06/27 11:15:59 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 11:15:59 DEBUG : Creating backend with remote "remote:test_bucket/test4"
2024/06/27 11:15:59 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 11:15:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:59 DEBUG : HTTP REQUEST (req 0xc0005290e0)
2024/06/27 11:15:59 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 11:15:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:59 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:59 DEBUG : HTTP RESPONSE (req 0xc0005290e0)
2024/06/27 11:15:59 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:15:59 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 11:15:59 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:15:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:15:59 DEBUG : HTTP REQUEST (req 0xc0008c9d40)
2024/06/27 11:15:59 DEBUG : GET /storage/v1/b/test_bucket/o/test4?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:15:59 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:00 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:00 DEBUG : HTTP RESPONSE (req 0xc0008c9d40)
2024/06/27 11:16:00 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:15:59 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NpqdLbbUZ9qvP7A_4Y8GB9xSyoBvAgyc0b20iN5Pg7fzNdRRPhhcw3nHA4ojKWd1gC95JKgFv5ywg

2024/06/27 11:16:00 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:00 ERROR : Ignoring --no-traverse with sync
2024/06/27 11:16:00 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:00 DEBUG : HTTP REQUEST (req 0xc0003a4900)
2024/06/27 11:16:00 DEBUG : GET /storage/v1/b/test_bucket/o?alt=json&maxResults=1000&prefix=test4%2F&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:16:00 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:00 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:00 DEBUG : HTTP RESPONSE (req 0xc0003a4900)
2024/06/27 11:16:00 DEBUG : HTTP/2.0 200 OK
Content-Length: 821
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:16:00 GMT
Expires: Thu, 27 Jun 2024 14:16:00 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NpAJbJ2eopDMV0a_LraQmUrkRMA25CWotJr1yT6pa8fCqfqWmpAD5KcH4vukXHLUsQfKacthx3HrA

2024/06/27 11:16:00 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:00 DEBUG : GCS bucket test_bucket path test4: Waiting for checks to finish
2024/06/27 11:16:00 DEBUG : test4: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/06/27 11:16:00 DEBUG : test4: Unchanged skipping
2024/06/27 11:16:00 DEBUG : GCS bucket test_bucket path test4: Waiting for transfers to finish
2024/06/27 11:16:00 DEBUG : Waiting for deletions to finish
2024/06/27 11:16:00 INFO  : There was nothing to transfer
2024/06/27 11:16:00 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         0.7s

2024/06/27 11:16:00 DEBUG : 7 go routines active

copy if the file exists on the remote:

2024/06/27 11:16:31 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "/local/test5" "remote:test_bucket/test5" "--no-traverse" "--fast-list" "--config" "/etc/rclone.conf" "--dump=headers" "-vv"]
2024/06/27 11:16:31 DEBUG : Creating backend with remote "/local/test5"
2024/06/27 11:16:31 DEBUG : Using config file from "/etc/rclone.conf"
2024/06/27 11:16:31 DEBUG : Creating backend with remote "remote:test_bucket/test5"
2024/06/27 11:16:31 DEBUG : You have specified to dump information. Please be noted that the Accept-Encoding as shown may not be correct in the request and the response may not show Content-Encoding if the go standard libraries auto gzip encoding was in effect. In this case the body of the request will be gunzipped before showing it.
2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:31 DEBUG : HTTP REQUEST (req 0xc00058f7a0)
2024/06/27 11:16:31 DEBUG : POST /token HTTP/1.1
Host: oauth2.googleapis.com
User-Agent: rclone/v1.67.0
Content-Length: 800
Content-Type: application/x-www-form-urlencoded
Accept-Encoding: gzip

2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:31 DEBUG : HTTP RESPONSE (req 0xc00058f7a0)
2024/06/27 11:16:31 DEBUG : HTTP/2.0 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:16:31 GMT
Server: scaffolding on HTTPServer2
Vary: Origin
Vary: X-Origin
Vary: Referer
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 0

2024/06/27 11:16:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:31 DEBUG : HTTP REQUEST (req 0xc000664000)
2024/06/27 11:16:31 DEBUG : GET /storage/v1/b/test_bucket/o/test5?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:31 DEBUG : HTTP RESPONSE (req 0xc000664000)
2024/06/27 11:16:31 DEBUG : HTTP/2.0 404 Not Found
Content-Length: 169
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:16:31 GMT
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Pragma: no-cache
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0NqG_3YtnUTLjOizaRaFlOcNUvqBT_xLg2ci2fvlHDTRPButIRMP86FZ0CrWikEGBaszbmYABKixhw

2024/06/27 11:16:31 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:31 DEBUG : HTTP REQUEST (req 0xc000b26480)
2024/06/27 11:16:31 DEBUG : GET /storage/v1/b/test_bucket/o/test5%2Ftest5?alt=json&prettyPrint=false HTTP/1.1
Host: storage.googleapis.com
User-Agent: rclone/v1.67.0
Authorization: XXXX
X-Goog-Api-Client: gl-go/1.22.4 gdcl/0.168.0
Accept-Encoding: gzip

2024/06/27 11:16:31 DEBUG : >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2024/06/27 11:16:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:32 DEBUG : HTTP RESPONSE (req 0xc000b26480)
2024/06/27 11:16:32 DEBUG : HTTP/2.0 200 OK
Content-Length: 784
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Cache-Control: private, max-age=0, must-revalidate, no-transform
Content-Type: application/json; charset=UTF-8
Date: Thu, 27 Jun 2024 14:16:32 GMT
Etag: CL+RqPD3+4YDEAE=
Expires: Thu, 27 Jun 2024 14:16:32 GMT
Server: UploadServer
Vary: Origin
Vary: X-Origin
X-Guploader-Uploadid: ACJd0Nrlp5zlHIJSVsq0sdw2-30y1ZK9aqrcRIgmadl0LV2FIlLEGKk2E5h5y1zPN3FwLcyNW7pGJDPrnw

2024/06/27 11:16:32 DEBUG : <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
2024/06/27 11:16:32 DEBUG : GCS bucket test_bucket path test5: Waiting for checks to finish
2024/06/27 11:16:32 DEBUG : test5: Size and modification time the same (differ by 0s, within tolerance 1ns)
2024/06/27 11:16:32 DEBUG : test5: Unchanged skipping
2024/06/27 11:16:32 DEBUG : GCS bucket test_bucket path test5: Waiting for transfers to finish
2024/06/27 11:16:32 INFO  : There was nothing to transfer
2024/06/27 11:16:32 INFO  :
Transferred:              0 B / 0 B, -, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Elapsed time:         1.1s

2024/06/27 11:16:32 DEBUG : 7 go routines active

cberni · June 27, 2024, 2:22pm

rclone move seems to have an additional check compared to rclone copy and sync.

asdffdsa · June 27, 2024, 5:05pm

checks are internal to rclone, does not correspond to api calls.

let's focus on api calls.
since all logs have the same POST /token, let's ignore that.

conclusion: each log has two GET api calls

system · July 27, 2024, 5:05pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.