Rclone Copy vs Sync: Test Results

I did some tests yesterday to compare rclone copy vs sync.
Here are the results

rclone v1.53.1
- os/arch: linux/amd64
- go version: go1.15

Source OS: CentOS 8
Source Host: Linode
Source CPU: Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Source RAM: 2GB

Destination: Google Drive

Transferred Files & Folders:
Total objects: 34762
Total size: 3.382 GBytes (3631235119 Bytes)

Sync Time: 09:30:05 - 12:22:04 = 172 min
Copy Time: 12:55:04 - 19:01:09 = 366 min

Hope it helps.

Without any more information, that's really just some numbers as copy and sync are two different things and it's not quite comparing the same things.

What are you trying to show?
What's the actual commands you ran?

I thought it was evident.
--> The difference in time taken between sync & copy.
I wanted to get an estimate of the difference in time taken for replicating files between source and destination via copy & sync.

I thought sync will consume more time and less bandwidth as compared to copy but as it turns out, in my case, copy took way more time than sync.

rclone sync source destination
rclone copy source destination

Sync and copy do two different things though so what's the point of measuring them?

If you are running the same source to a clean destination, they'd take roughly the same time and would not be much different.

You'd have to run both and share a debug log to see what the difference is as your results don't make sense.

That was obvious to me. If there is nothing in the destination for sync to compare, its just copy everything, which is the same as copy.

@Animosity022
I don't have a question here & I'm not looking for any answers from you or anyone in the forum.
I did a test to decide if I should use sync or copy based on time takes and b/w used -- I have constraints on both, time & b/w from a hosted infra perspective. I understand the end goal of both commands is same.

It's fine if my logs do not make sense to you. Its did to me & I shared it here if anyone else has a similar decision to make they may have some data to start with.

If you believe my stats may be misleading or incorrect or incomplete or anything adverse, please feel free to delete my post or let me know I will be happy to do so myself.

A sync and copy use the same routines so if you are doing either from the same source to a fresh destination, the results would be the marginally the same. Your results don't seem clear so I don't want someone to draw the wrong conclusion from your data as it doesn't seem accurate, hence the request for logs and more information.

I'm asking questions to try to quantify your statements.

you did not mention that in first post.

if you can better define the parameters of two tests, that would be helpful to understand the reason for the difference in total times.

you have not posted the commands used for testing?

did you use a debug log and look for errors, perhaps there was a glitch with the internet or gdrive?

given the large number of small files transferred and the limits of gdrive in terms of files transferred per second, i would guess that the difference in total time would be something external to rclone itself.
without more details, it is just a guess

i agree with everyone here.

to me it's still not completely clear if the first test was done to an empty destination, as much as that's exactly what i would expect from such a test.

and, on my brief experience with rclone, this feels right. sync surely seem to be much faster than copy, for whatever reason.

sorry i probably won't be able to do much benchmarking and testing on this, though.

Sync is slower as it has to compare source to destination and delete on the destination so it has more operations to do.

When a file is "copied" via copy or sync, they are using the same routines so they perform identical.

not talking about theory, though. :wink:

A copy does one thing. It copies.

A sync does two things. It copies and deletes on the destination to make the destination resemble the source.

Doing two things take longer than one thing. No theory about it :slight_smile:

Nothing about theory here.

his practical test and my practical (albeit too brief) experience tells another story.

i would be very surprised if there are absolutely no more practical instances of the case in which sync doesn't go much faster.

That's anecdotal evidence as both tests aren't very accurate results.

Validate your API hits via Google Drive and you'll see more for the sync since it's doing more and will take longer if all the your variables are the same.

We're debating of running to first base is faster than running to second base as to sync, you get that extra base in and regardless of the test, it always takes longer to run to two bases rather than one since they are two operation happening.

Less files skew the test as there is less difference.
More files would show a better use case of a real world test.

Google inherently is bad as copying many small files are you can only get 2-3 per second and there are so few details what actually was executed and zero logs, it's practically useless in terms of showing a repeatable copy vs sync performance.

1 Like

there are at least 3 possible debates here.

  • you're going full theoretical, i'm not debating this. no idea how rclone should work on the background, but it still appears to me you're oversimplifying it. i do know, for instance, there are settings to build an index of files in different moments. a complex piece of code like rclone surely have more details that could easily be overseen. again, though, i have zero wish to debate this.

  • we could debate on the practical realm, but nobody is doing more tests. so i think we both have zero wishes here!

  • the actual debate i can see is only around english language and concepts, while better trying to understand each other.

thus we're both mostly being redundant. you are at least slowly adding some theoretical points, i am on the other hand contributing with much less, by only adding english redundancy.

i guess my will initially was just to share an extra data point to the speed test, but now it looks like it was of nobody's interest. it was never my intention to debate anything, yet i did engage in it a bit too much.

and now i think this is my limit, so i'll abstain myself from doing more of this.

I have plenty of tests as I copy a lot and use sync and over a year's worth of logs / data.

The problem with Google Drive is a bad 'file operation' skews a whole set of data as if you transfer a 100 files and 1 file gets a bad copy rate, it makes the whole data set look bad.

An example would be something like:

felix@gemini:~/test$ rclone copy /home/felix/test GD:test1 -P
Transferred:         1000M / 1000 MBytes, 100%, 14.201 MBytes/s, ETA 0s
Transferred:           10 / 10, 100%
Elapsed time:      1m10.8s

vs

felix@gemini:~/test$ rclone sync /home/felix/test GD:test3 -P
Transferred:         1000M / 1000 MBytes, 100%, 63.732 MBytes/s, ETA 0s
Transferred:           10 / 10, 100%
Elapsed time:        21.6s

Sync is faster, right? The issue there is I had 1 file that got a slow transfer speed so it skews the result of the test.

With a small data set in the real world, a copy and sync would take almost identical times since it's only comparing 10 files as the end to see if it needs to delete anything.

If I have 30k files, we are adding more time.

Engaging in discussion and trying to understand how things work is superb and how we all learn.

I've been right and wrong and I'm sure that will continue throughout as I ask a lot of question and challenge things.

In this case as I'd like to say at work, I'd bet a paycheck a sync has more overhead and generally takes longer than running just a copy command.

That's always a choice you can make but I really thought it was a healthy discussion from my perspective.

well, since you're enjoying it so much AND so willing to do tests, i can try once more for today...

simply put, the more randomised* tests there are, the better to be sure.

if it's all about google, just do more tests outside of google.

ideally there would be an option under the configuration to contribute with anonymous data for things like this.

perhaps you know better than i, there is a variance expected for something like sync being faster than copy, unexpectedly like this... in general i think that's like 20% of the time, so if it is indeed a random event that's causing this unexpected result, it would be rather normal to see it happening by pure chance 1 out of 5 times, with a big enough dataset. not sure this is the number or this is the case, though! it's just hopefully good to have in mind that randomness* is both unexpected and still somewhat measurable.

* and in both cases marked with an asterisk, i'm talking about different kinds of random. there are too many kinds of random and chaos... much more than the 20 or 30 different kinds of snow that get different names by eskimos. but we don't (or i certainly don't) have names for them.

since my sync and dedupe takes almost always less a bit than 1h (except once, the very first time, it took 2h) and the only copy i've tried so far is taking over 4h, i would be slightly interested to know if i should keep using only sync.

at the same time, i hope i won't be needing or using this gdrive for much longer this year. if even once more.

out to rest/sleep now. :kissing_heart:

1 Like

That depends on what is random. I can control much of my test minus what Google actually does. If you are maxing out upload/download/CPU/things that are inside of one's control, that makes the the data worse and skews the results as well. The goal would be to test how Google reacts to changes/parameters/etc while removing out the client side of that and making it more 'stable'.

My backend experience/testing/months/years of data comes from Google Drive as I don't use other backends.

I have monitoring to measure my home bandwidth, all normal hot spots on my Linux machine and lots and lots of data.

Some daily examples:

Transferred:       98.216G / 98.216 GBytes, 100%, 86.805 MBytes/s, ETA 0s
Transferred:       76.967G / 76.967 GBytes, 100%, 71.005 MBytes/s, ETA 0s
Transferred:      190.724G / 190.724 GBytes, 100%, 91.680 MBytes/s, ETA 0s
Transferred:      372.374G / 372.374 GBytes, 100%, 84.925 MBytes/s, ETA 0s
Transferred:      302.877G / 302.877 GBytes, 100%, 77.271 MBytes/s, ETA 0s
Transferred:      606.786G / 606.786 GBytes, 100%, 95.389 MBytes/s, ETA 0s
Transferred:      211.159G / 211.159 GBytes, 100%, 79.738 MBytes/s, ETA 0s
Transferred:      509.103G / 509.103 GBytes, 100%, 99.650 MBytes/s, ETA 0s
Transferred:      156.988G / 156.988 GBytes, 100%, 85.395 MBytes/s, ETA 0s
Transferred:      331.485G / 331.485 GBytes, 100%, 92.415 MBytes/s, ETA 0s
Transferred:      225.617G / 225.617 GBytes, 100%, 99.859 MBytes/s, ETA 0s
Transferred:      234.787G / 234.787 GBytes, 100%, 87.199 MBytes/s, ETA 0s
Transferred:      142.146G / 142.146 GBytes, 100%, 77.834 MBytes/s, ETA 0s
Transferred:      122.509G / 122.509 GBytes, 100%, 72.477 MBytes/s, ETA 0s
Transferred:      300.149G / 300.149 GBytes, 100%, 80.770 MBytes/s, ETA 0s
Transferred:      382.505G / 382.505 GBytes, 100%, 95.846 MBytes/s, ETA 0s
Transferred:       60.208G / 60.208 GBytes, 100%, 95.960 MBytes/s, ETA 0s
Transferred:      217.346G / 217.346 GBytes, 100%, 91.004 MBytes/s, ETA 0s
Transferred:      142.373G / 142.373 GBytes, 100%, 89.002 MBytes/s, ETA 0s
Transferred:      102.893G / 102.893 GBytes, 100%, 69.188 MBytes/s, ETA 0s
Transferred:      123.427G / 123.427 GBytes, 100%, 88.992 MBytes/s, ETA 0s
Transferred:      148.590G / 148.590 GBytes, 100%, 81.589 MBytes/s, ETA 0s
Transferred:      277.503G / 277.503 GBytes, 100%, 93.611 MBytes/s, ETA 0s
Transferred:      477.132G / 477.132 GBytes, 100%, 89.420 MBytes/s, ETA 0s
Transferred:       56.672G / 56.672 GBytes, 100%, 81.899 MBytes/s, ETA 0s
Transferred:       78.948G / 78.948 GBytes, 100%, 75.752 MBytes/s, ETA 0s
Transferred:      130.869G / 130.931 GBytes, 100%, 82.107 MBytes/s, ETA 0s
Transferred:      130.936G / 130.936 GBytes, 100%, 81.988 MBytes/s, ETA 0s
Transferred:      256.354G / 256.354 GBytes, 100%, 89.715 MBytes/s, ETA 0s
Transferred:      128.726G / 128.726 GBytes, 100%, 62.104 MBytes/s, ETA 0s
Transferred:       65.498G / 65.498 GBytes, 100%, 78.713 MBytes/s, ETA 0s
Transferred:      146.199G / 146.199 GBytes, 100%, 83.889 MBytes/s, ETA 0s
Transferred:      136.844G / 136.844 GBytes, 100%, 78.534 MBytes/s, ETA 0s
Transferred:       59.605G / 59.605 GBytes, 100%, 87.850 MBytes/s, ETA 0s
Transferred:       33.083G / 33.083 GBytes, 100%, 74.338 MBytes/s, ETA 0s
Transferred:      118.189G / 118.189 GBytes, 100%, 76.649 MBytes/s, ETA 0s
Transferred:       36.668G / 36.668 GBytes, 100%, 87.283 MBytes/s, ETA 0s
Transferred:       84.163G / 84.163 GBytes, 100%, 89.643 MBytes/s, ETA 0s
Transferred:      112.651G / 112.651 GBytes, 100%, 82.999 MBytes/s, ETA 0s
Transferred:       96.698G / 96.698 GBytes, 100%, 77.861 MBytes/s, ETA 0s
Transferred:       81.098G / 81.098 GBytes, 100%, 93.405 MBytes/s, ETA 0s
Transferred:      176.089G / 176.089 GBytes, 100%, 80.911 MBytes/s, ETA 0s
Transferred:       56.398G / 56.398 GBytes, 100%, 66.109 MBytes/s, ETA 0s
Transferred:      196.707G / 196.707 GBytes, 100%, 89.157 MBytes/s, ETA 0s
Transferred:      200.488G / 200.488 GBytes, 100%, 81.759 MBytes/s, ETA 0s
Transferred:      103.608G / 103.608 GBytes, 100%, 81.989 MBytes/s, ETA 0s
Transferred:       95.446G / 95.446 GBytes, 100%, 84.010 MBytes/s, ETA 0s
Transferred:      177.573G / 177.573 GBytes, 100%, 92.776 MBytes/s, ETA 0s
Transferred:       93.904G / 93.904 GBytes, 100%, 91.718 MBytes/s, ETA 0s
Transferred:      366.946G / 366.946 GBytes, 100%, 86.300 MBytes/s, ETA 0s
Transferred:      191.993G / 191.993 GBytes, 100%, 88.376 MBytes/s, ETA 0s
Transferred:      192.641G / 192.641 GBytes, 100%, 83.044 MBytes/s, ETA 0s
Transferred:      103.612G / 103.612 GBytes, 100%, 84.977 MBytes/s, ETA 0s
Transferred:       90.996G / 90.996 GBytes, 100%, 93.988 MBytes/s, ETA 0s
Transferred:      170.017G / 170.017 GBytes, 100%, 83.908 MBytes/s, ETA 0s
Transferred:      338.913G / 338.913 GBytes, 100%, 82.235 MBytes/s, ETA 0s
Transferred:      306.551G / 306.551 GBytes, 100%, 90.147 MBytes/s, ETA 0s
Transferred:      610.965G / 610.965 GBytes, 100%, 78.505 MBytes/s, ETA 0s
Transferred:       93.455G / 93.455 GBytes, 100%, 79.798 MBytes/s, ETA 0s
Transferred:      235.478G / 235.478 GBytes, 100%, 86.366 MBytes/s, ETA 0s
Transferred:      188.914G / 189.814 GBytes, 100%, 84.754 MBytes/s, ETA 10s
Transferred:      189.818G / 189.818 GBytes, 100%, 84.597 MBytes/s, ETA 0s
Transferred:      380.468G / 380.468 GBytes, 100%, 84.535 MBytes/s, ETA 0s
Transferred:      110.434G / 110.434 GBytes, 100%, 84.253 MBytes/s, ETA 0s
Transferred:      184.299G / 184.299 GBytes, 100%, 82.360 MBytes/s, ETA 0s
Transferred:       86.269G / 86.269 GBytes, 100%, 78.021 MBytes/s, ETA 0s
Transferred:       86.269G / 86.269 GBytes, 100%, 78.021 MBytes/s, ETA 0s
Transferred:      230.493G / 230.493 GBytes, 100%, 87.727 MBytes/s, ETA 0s
Transferred:      206.242G / 206.242 GBytes, 100%, 88.565 MBytes/s, ETA 0s
Transferred:      313.967G / 313.967 GBytes, 100%, 84.012 MBytes/s, ETA 0s
Transferred:      408.996G / 409.034 GBytes, 100%, 88.297 MBytes/s, ETA 0s
Transferred:      409.036G / 409.036 GBytes, 100%, 88.268 MBytes/s, ETA 0s
Transferred:      326.917G / 326.917 GBytes, 100%, 86.526 MBytes/s, ETA 0s
Transferred:      585.188G / 585.188 GBytes, 100%, 39.312 MBytes/s, ETA 0s
Transferred:        7.603G / 7.603 GBytes, 100%, 41.883 MBytes/s, ETA 0s
Transferred:      270.277G / 270.277 GBytes, 100%, 40.108 MBytes/s, ETA 0s
Transferred:       74.532G / 74.532 GBytes, 100%, 43.516 MBytes/s, ETA 0s
Transferred:      172.446G / 172.446 GBytes, 100%, 42.753 MBytes/s, ETA 0s
Transferred:       71.899G / 71.899 GBytes, 100%, 37.371 MBytes/s, ETA 0s
Transferred:       30.223G / 30.223 GBytes, 100%, 40.821 MBytes/s, ETA 0s
Transferred:      221.487G / 221.487 GBytes, 100%, 35.596 MBytes/s, ETA 0s
Transferred:      222.471G / 222.471 GBytes, 100%, 35.602 MBytes/s, ETA 0s
Transferred:      101.951G / 101.951 GBytes, 100%, 27.863 MBytes/s, ETA 0s
Transferred:      168.295G / 168.295 GBytes, 100%, 41.474 MBytes/s, ETA 0s
Transferred:       52.768G / 52.768 GBytes, 100%, 86.113 MBytes/s, ETA 0s
Transferred:       69.791G / 69.791 GBytes, 100%, 80.176 MBytes/s, ETA 0s
Transferred:      380.333G / 380.333 GBytes, 100%, 84.716 MBytes/s, ETA 0s
Transferred:      521.013G / 521.013 GBytes, 100%, 81.907 MBytes/s, ETA 0s
Transferred:      229.990G / 229.990 GBytes, 100%, 84.849 MBytes/s, ETA 0s
Transferred:      244.802G / 244.802 GBytes, 100%, 83.528 MBytes/s, ETA 0s
Transferred:       98.521G / 98.521 GBytes, 100%, 81.753 MBytes/s, ETA 0s
Transferred:      135.245G / 135.245 GBytes, 100%, 77.785 MBytes/s, ETA 0s
Transferred:      177.508G / 177.508 GBytes, 100%, 78.219 MBytes/s, ETA 0s
Transferred:      168.368G / 168.412 GBytes, 100%, 79.857 MBytes/s, ETA 0s
Transferred:      168.414G / 168.414 GBytes, 100%, 79.682 MBytes/s, ETA 0s
Transferred:      590.940G / 593.172 GBytes, 100%, 74.647 MBytes/s, ETA 30s
Transferred:      593.178G / 593.178 GBytes, 100%, 73.608 MBytes/s, ETA 0s
Transferred:      334.800G / 334.800 GBytes, 100%, 84.398 MBytes/s, ETA 0s
Transferred:      456.221G / 456.221 GBytes, 100%, 86.639 MBytes/s, ETA 0s
Transferred:      183.628G / 183.628 GBytes, 100%, 87.146 MBytes/s, ETA 0s
Transferred:      183.628G / 183.628 GBytes, 100%, 87.146 MBytes/s, ETA 0s
Transferred:      411.686G / 411.686 GBytes, 100%, 80.833 MBytes/s, ETA 0s
Transferred:      411.686G / 411.686 GBytes, 100%, 80.833 MBytes/s, ETA 0s
Transferred:      415.344G / 415.344 GBytes, 100%, 80.828 MBytes/s, ETA 0s
Transferred:      212.884G / 212.884 GBytes, 100%, 92.027 MBytes/s, ETA 0s
Transferred:      384.992G / 384.992 GBytes, 100%, 89.125 MBytes/s, ETA 0s
Transferred:       48.035G / 48.035 GBytes, 100%, 86.058 MBytes/s, ETA 0s
Transferred:       61.570G / 61.570 GBytes, 100%, 82.801 MBytes/s, ETA 0s
Transferred:       63.246G / 63.246 GBytes, 100%, 81.620 MBytes/s, ETA 0s
Transferred:      412.908G / 412.908 GBytes, 100%, 85.889 MBytes/s, ETA 0s
Transferred:       54.635G / 54.635 GBytes, 100%, 86.967 MBytes/s, ETA 0s
Transferred:      246.518G / 246.518 GBytes, 100%, 88.866 MBytes/s, ETA 0s
Transferred:      228.247G / 228.247 GBytes, 100%, 91.020 MBytes/s, ETA 0s
Transferred:      137.731G / 137.731 GBytes, 100%, 81.488 MBytes/s, ETA 0s
Transferred:       98.270G / 98.270 GBytes, 100%, 91.255 MBytes/s, ETA 0s
Transferred:      105.750G / 105.800 GBytes, 100%, 78.889 MBytes/s, ETA 0s
Transferred:      105.804G / 105.804 GBytes, 100%, 78.841 MBytes/s, ETA 0s
Transferred:      353.146G / 353.146 GBytes, 100%, 91.344 MBytes/s, ETA 0s
Transferred:       73.420G / 73.420 GBytes, 100%, 84.683 MBytes/s, ETA 0s
Transferred:      120.573G / 120.573 GBytes, 100%, 84.476 MBytes/s, ETA 0s
Transferred:      177.498G / 177.498 GBytes, 100%, 84.131 MBytes/s, ETA 0s
Transferred:      148.505G / 148.505 GBytes, 100%, 90.667 MBytes/s, ETA 0s
Transferred:      233.765G / 233.765 GBytes, 100%, 90.629 MBytes/s, ETA 0s
Transferred:      252.485G / 252.485 GBytes, 100%, 89.595 MBytes/s, ETA 0s
Transferred:      246.547G / 246.547 GBytes, 100%, 92.010 MBytes/s, ETA 0s
Transferred:      373.924G / 373.924 GBytes, 100%, 88.084 MBytes/s, ETA 0s
Transferred:       82.736G / 82.736 GBytes, 100%, 83.872 MBytes/s, ETA 0s
Transferred:      548.857G / 548.857 GBytes, 100%, 91.549 MBytes/s, ETA 0s
Transferred:      649.030G / 651.807 GBytes, 100%, 92.487 MBytes/s, ETA 30s
Transferred:      651.810G / 651.810 GBytes, 100%, 92.399 MBytes/s, ETA 0s
Transferred:      182.725G / 182.725 GBytes, 100%, 92.536 MBytes/s, ETA 0s
Transferred:      382.715G / 382.715 GBytes, 100%, 84.467 MBytes/s, ETA 0s
Transferred:      178.965G / 178.965 GBytes, 100%, 91.270 MBytes/s, ETA 0s
Transferred:      186.780G / 186.780 GBytes, 100%, 87.152 MBytes/s, ETA 0s
Transferred:      396.251G / 396.251 GBytes, 100%, 90.636 MBytes/s, ETA 0s
Transferred:       88.772G / 88.772 GBytes, 100%, 89.398 MBytes/s, ETA 0s
Transferred:      168.234G / 168.234 GBytes, 100%, 92.473 MBytes/s, ETA 0s
Transferred:      362.903G / 363.881 GBytes, 100%, 92.281 MBytes/s, ETA 10s
Transferred:      363.883G / 363.883 GBytes, 100%, 92.225 MBytes/s, ETA 0s
Transferred:       74.738G / 74.738 GBytes, 100%, 89.533 MBytes/s, ETA 0s
Transferred:      262.336G / 262.336 GBytes, 100%, 91.200 MBytes/s, ETA 0s
Transferred:       75.633G / 75.633 GBytes, 100%, 89.587 MBytes/s, ETA 0s
Transferred:      421.078G / 421.078 GBytes, 100%, 91.730 MBytes/s, ETA 0s
Transferred:      233.868G / 233.868 GBytes, 100%, 91.409 MBytes/s, ETA 0s
Transferred:      465.253G / 467.017 GBytes, 100%, 90.260 MBytes/s, ETA 20s
Transferred:      467.019G / 467.019 GBytes, 100%, 90.218 MBytes/s, ETA 0s
Transferred:      145.058G / 145.058 GBytes, 100%, 82.360 MBytes/s, ETA 0s
Transferred:       26.484G / 26.484 GBytes, 100%, 65.611 MBytes/s, ETA 0s
Transferred:      225.894G / 225.894 GBytes, 100%, 87.115 MBytes/s, ETA 0s
Transferred:       52.503G / 52.503 GBytes, 100%, 85.592 MBytes/s, ETA 0s
Transferred:      199.713G / 199.713 GBytes, 100%, 90.489 MBytes/s, ETA 0s
Transferred:        9.881G / 9.881 GBytes, 100%, 87.609 MBytes/s, ETA 0s
Transferred:      117.547G / 117.547 GBytes, 100%, 85.896 MBytes/s, ETA 0s
Transferred:       44.273G / 44.273 GBytes, 100%, 73.434 MBytes/s, ETA 0s
Transferred:       92.072G / 92.072 GBytes, 100%, 90.905 MBytes/s, ETA 0s
Transferred:      302.985G / 302.985 GBytes, 100%, 90.603 MBytes/s, ETA 0s
Transferred:      158.684G / 158.684 GBytes, 100%, 87.134 MBytes/s, ETA 0s
Transferred:       66.934G / 66.934 GBytes, 100%, 88.787 MBytes/s, ETA 0s
Transferred:       85.986G / 85.986 GBytes, 100%, 85.597 MBytes/s, ETA 0s
Transferred:      592.823G / 592.823 GBytes, 100%, 82.934 MBytes/s, ETA 0s
Transferred:      128.754G / 128.754 GBytes, 100%, 90.485 MBytes/s, ETA 0s
Transferred:      230.465G / 230.465 GBytes, 100%, 87.641 MBytes/s, ETA 0s
Transferred:       62.056G / 62.056 GBytes, 100%, 86.376 MBytes/s, ETA 0s
Transferred:       59.540G / 59.540 GBytes, 100%, 86.672 MBytes/s, ETA 0s
Transferred:      135.935G / 135.935 GBytes, 100%, 89.425 MBytes/s, ETA 0s
Transferred:      531.814G / 531.814 GBytes, 100%, 86.570 MBytes/s, ETA 0s
Transferred:      871.586M / 871.586 MBytes, 100%, 68.737 MBytes/s, ETA 0s
Transferred:       63.515G / 63.515 GBytes, 100%, 78.469 MBytes/s, ETA 0s
Transferred:      669.868G / 669.868 GBytes, 100%, 93.272 MBytes/s, ETA 0s
Transferred:       27.213G / 27.213 GBytes, 100%, 91.166 MBytes/s, ETA 0s
Transferred:       83.660G / 83.660 GBytes, 100%, 93.358 MBytes/s, ETA 0s
Transferred:      205.190G / 205.190 GBytes, 100%, 91.943 MBytes/s, ETA 0s
Transferred:       89.245G / 89.245 GBytes, 100%, 89.191 MBytes/s, ETA 0s
Transferred:      106.458G / 106.458 GBytes, 100%, 90.866 MBytes/s, ETA 0s
Transferred:      120.841G / 120.841 GBytes, 100%, 93.034 MBytes/s, ETA 0s
Transferred:      146.707G / 146.707 GBytes, 100%, 91.654 MBytes/s, ETA 0s
Transferred:       94.632G / 94.632 GBytes, 100%, 92.664 MBytes/s, ETA 0s
Transferred:      170.293G / 170.293 GBytes, 100%, 90.860 MBytes/s, ETA 0s
Transferred:      203.345G / 203.345 GBytes, 100%, 91.676 MBytes/s, ETA 0s
Transferred:       62.755G / 62.755 GBytes, 100%, 88.528 MBytes/s, ETA 0s
Transferred:      152.575G / 152.575 GBytes, 100%, 92.051 MBytes/s, ETA 0s
Transferred:       86.819G / 86.819 GBytes, 100%, 87.599 MBytes/s, ETA 0s
Transferred:      269.769G / 269.769 GBytes, 100%, 94.290 MBytes/s, ETA 0s
Transferred:      130.498G / 130.498 GBytes, 100%, 94.307 MBytes/s, ETA 0s
Transferred:      174.574G / 174.574 GBytes, 100%, 91.840 MBytes/s, ETA 0s
Transferred:       26.581G / 26.581 GBytes, 100%, 82.192 MBytes/s, ETA 0s
Transferred:      185.576G / 186.404 GBytes, 100%, 80.997 MBytes/s, ETA 10s
Transferred:      186.415G / 186.415 GBytes, 100%, 80.737 MBytes/s, ETA 0s
Transferred:      117.905G / 117.905 GBytes, 100%, 78.416 MBytes/s, ETA 0s
Transferred:       68.475G / 68.475 GBytes, 100%, 91.993 MBytes/s, ETA 0s
Transferred:      323.970G / 323.970 GBytes, 100%, 93.806 MBytes/s, ETA 0s
Transferred:      366.353G / 366.353 GBytes, 100%, 91.422 MBytes/s, ETA 0s
Transferred:      239.615G / 239.615 GBytes, 100%, 91.707 MBytes/s, ETA 0s
Transferred:      727.791G / 727.791 GBytes, 100%, 91.207 MBytes/s, ETA 0s
Transferred:      175.777G / 175.777 GBytes, 100%, 85.808 MBytes/s, ETA 0s
Transferred:       17.119G / 17.119 GBytes, 100%, 78.876 MBytes/s, ETA 0s
Transferred:       92.843G / 92.843 GBytes, 100%, 88.690 MBytes/s, ETA 0s
Transferred:      134.969G / 134.969 GBytes, 100%, 89.226 MBytes/s, ETA 0s
Transferred:      120.588G / 120.588 GBytes, 100%, 89.681 MBytes/s, ETA 0s
Transferred:      548.773G / 548.773 GBytes, 100%, 95.740 MBytes/s, ETA 0s
Transferred:      169.452G / 169.452 GBytes, 100%, 97.474 MBytes/s, ETA 0s
Transferred:      142.011G / 142.011 GBytes, 100%, 87.601 MBytes/s, ETA 0s
Transferred:       24.206G / 24.206 GBytes, 100%, 79.683 MBytes/s, ETA 0s
Transferred:      174.120G / 174.120 GBytes, 100%, 86.924 MBytes/s, ETA 0s
Transferred:      177.870G / 177.870 GBytes, 100%, 80.233 MBytes/s, ETA 0s
Transferred:      164.153G / 164.153 GBytes, 100%, 90.459 MBytes/s, ETA 0s
Transferred:      357.907G / 357.992 GBytes, 100%, 96.762 MBytes/s, ETA 0s
Transferred:      357.995G / 357.995 GBytes, 100%, 96.713 MBytes/s, ETA 0s
Transferred:      296.082G / 296.082 GBytes, 100%, 88.790 MBytes/s, ETA 0s
Transferred:      387.204G / 387.204 GBytes, 100%, 91.774 MBytes/s, ETA 0s
Transferred:       80.987G / 80.987 GBytes, 100%, 88.339 MBytes/s, ETA 0s
Transferred:      132.042G / 132.042 GBytes, 100%, 94.384 MBytes/s, ETA 0s
Transferred:       60.501G / 60.501 GBytes, 100%, 97.990 MBytes/s, ETA 0s
Transferred:      102.227G / 102.227 GBytes, 100%, 93.315 MBytes/s, ETA 0s
Transferred:      105.485G / 105.485 GBytes, 100%, 89.436 MBytes/s, ETA 0s
Transferred:       87.430G / 87.430 GBytes, 100%, 90.718 MBytes/s, ETA 0s
Transferred:       98.722G / 98.722 GBytes, 100%, 95.725 MBytes/s, ETA 0s
Transferred:      198.399G / 198.399 GBytes, 100%, 98.004 MBytes/s, ETA 0s
Transferred:      186.174G / 186.174 GBytes, 100%, 93.514 MBytes/s, ETA 0s
Transferred:       56.515G / 56.515 GBytes, 100%, 61.529 MBytes/s, ETA 0s
Transferred:       34.101G / 34.101 GBytes, 100%, 86.275 MBytes/s, ETA 0s
Transferred:      487.817G / 487.817 GBytes, 100%, 88.889 MBytes/s, ETA 0s
Transferred:      140.771G / 140.771 GBytes, 100%, 92.724 MBytes/s, ETA 0s
Transferred:       90.923G / 90.923 GBytes, 100%, 96.860 MBytes/s, ETA 0s
Transferred:       35.007G / 35.007 GBytes, 100%, 74.740 MBytes/s, ETA 0s
Transferred:      109.182G / 109.182 GBytes, 100%, 78.204 MBytes/s, ETA 0s
Transferred:       75.536G / 75.536 GBytes, 100%, 84.604 MBytes/s, ETA 0s
Transferred:       11.508G / 11.508 GBytes, 100%, 72.871 MBytes/s, ETA 0s
Transferred:      142.663G / 142.663 GBytes, 100%, 92.828 MBytes/s, ETA 0s
Transferred:       36.551G / 36.551 GBytes, 100%, 91.360 MBytes/s, ETA 0s
Transferred:       35.648G / 35.648 GBytes, 100%, 93.040 MBytes/s, ETA 0s
Transferred:      126.515G / 126.515 GBytes, 100%, 91.770 MBytes/s, ETA 0s
Transferred:       38.630G / 38.630 GBytes, 100%, 89.837 MBytes/s, ETA 0s
Transferred:       19.876G / 19.876 GBytes, 100%, 79.331 MBytes/s, ETA 0s
Transferred:      434.904G / 434.904 GBytes, 100%, 95.949 MBytes/s, ETA 0s
Transferred:      356.598G / 356.598 GBytes, 100%, 62.983 MBytes/s, ETA 0s
Transferred:       82.348G / 82.348 GBytes, 100%, 93.798 MBytes/s, ETA 0s
Transferred:       40.813G / 40.813 GBytes, 100%, 97.368 MBytes/s, ETA 0s
Transferred:      192.956G / 192.956 GBytes, 100%, 97.450 MBytes/s, ETA 0s
Transferred:      140.519G / 140.519 GBytes, 100%, 93.894 MBytes/s, ETA 0s
Transferred:      155.463G / 155.463 GBytes, 100%, 95.766 MBytes/s, ETA 0s
Transferred:      136.914G / 136.914 GBytes, 100%, 96.772 MBytes/s, ETA 0s
Transferred:      113.609G / 113.609 GBytes, 100%, 97.937 MBytes/s, ETA 0s
Transferred:       85.694G / 85.694 GBytes, 100%, 91.881 MBytes/s, ETA 0s
Transferred:       77.261G / 77.261 GBytes, 100%, 91.501 MBytes/s, ETA 0s
Transferred:       89.154G / 89.154 GBytes, 100%, 98.687 MBytes/s, ETA 0s
Transferred:      363.073G / 363.073 GBytes, 100%, 96.228 MBytes/s, ETA 0s
Transferred:      196.560G / 196.560 GBytes, 100%, 99.020 MBytes/s, ETA 0s
Transferred:       32.250G / 32.250 GBytes, 100%, 87.003 MBytes/s, ETA 0s
Transferred:      206.370G / 206.370 GBytes, 100%, 97.091 MBytes/s, ETA 0s
Transferred:       91.547G / 91.547 GBytes, 100%, 93.252 MBytes/s, ETA 0s
Transferred:       13.754G / 13.754 GBytes, 100%, 82.898 MBytes/s, ETA 0s
Transferred:      131.483G / 131.483 GBytes, 100%, 94.821 MBytes/s, ETA 0s
Transferred:      313.680G / 313.680 GBytes, 100%, 97.241 MBytes/s, ETA 0s
Transferred:      118.723G / 118.723 GBytes, 100%, 96.655 MBytes/s, ETA 0s
Transferred:      255.457G / 255.457 GBytes, 100%, 95.053 MBytes/s, ETA 0s
Transferred:       34.184G / 34.184 GBytes, 100%, 76.419 MBytes/s, ETA 0s
Transferred:       69.287G / 69.287 GBytes, 100%, 89.063 MBytes/s, ETA 0s
Transferred:       32.534G / 32.534 GBytes, 100%, 90.680 MBytes/s, ETA 0s
Transferred:      181.109G / 181.109 GBytes, 100%, 92.538 MBytes/s, ETA 0s
Transferred:      139.050G / 139.050 GBytes, 100%, 96.864 MBytes/s, ETA 0s
Transferred:       86.810G / 86.810 GBytes, 100%, 97.771 MBytes/s, ETA 0s
Transferred:      164.680G / 164.680 GBytes, 100%, 98.567 MBytes/s, ETA 0s
Transferred:       96.711G / 96.711 GBytes, 100%, 94.671 MBytes/s, ETA 0s
Transferred:       37.660G / 37.660 GBytes, 100%, 89.896 MBytes/s, ETA 0s
Transferred:       35.144G / 35.144 GBytes, 100%, 89.411 MBytes/s, ETA 0s
Transferred:       60.388G / 60.388 GBytes, 100%, 95.423 MBytes/s, ETA 0s
Transferred:       38.121G / 38.121 GBytes, 100%, 92.478 MBytes/s, ETA 0s
Transferred:       70.991G / 70.991 GBytes, 100%, 95.624 MBytes/s, ETA 0s
Transferred:      219.864G / 219.864 GBytes, 100%, 85.816 MBytes/s, ETA 0s
Transferred:      164.672G / 164.672 GBytes, 100%, 93.601 MBytes/s, ETA 0s
Transferred:      297.865G / 297.865 GBytes, 100%, 94.104 MBytes/s, ETA 0s
Transferred:       95.743G / 95.743 GBytes, 100%, 95.370 MBytes/s, ETA 0s
Transferred:      110.143G / 110.143 GBytes, 100%, 94.406 MBytes/s, ETA 0s
Transferred:      271.540G / 271.540 GBytes, 100%, 98.388 MBytes/s, ETA 0s
Transferred:       28.066G / 28.066 GBytes, 100%, 83.830 MBytes/s, ETA 0s
Transferred:      136.300G / 136.300 GBytes, 100%, 89.828 MBytes/s, ETA 0s
Transferred:       42.808G / 42.833 GBytes, 100%, 90.365 MBytes/s, ETA 0s
Transferred:       42.836G / 42.836 GBytes, 100%, 89.590 MBytes/s, ETA 0s
Transferred:       48.439G / 48.439 GBytes, 100%, 93.171 MBytes/s, ETA 0s
Transferred:       29.623G / 29.623 GBytes, 100%, 91.880 MBytes/s, ETA 0s
Transferred:       18.757G / 18.757 GBytes, 100%, 92.104 MBytes/s, ETA 0s
Transferred:      257.128G / 257.128 GBytes, 100%, 93.653 MBytes/s, ETA 0s
Transferred:        9.855G / 9.855 GBytes, 100%, 81.279 MBytes/s, ETA 0s
Transferred:       64.804G / 64.804 GBytes, 100%, 93.724 MBytes/s, ETA 0s
Transferred:       68.856G / 68.856 GBytes, 100%, 85.538 MBytes/s, ETA 0s
Transferred:       76.362G / 76.362 GBytes, 100%, 87.352 MBytes/s, ETA 0s
Transferred:       30.070G / 30.070 GBytes, 100%, 85.814 MBytes/s, ETA 0s
Transferred:      114.301G / 114.301 GBytes, 100%, 90.386 MBytes/s, ETA 0s
Transferred:      225.170G / 225.170 GBytes, 100%, 82.252 MBytes/s, ETA 0s
Transferred:      165.815G / 165.815 GBytes, 100%, 94.307 MBytes/s, ETA 0s
Transferred:      333.151G / 333.151 GBytes, 100%, 97.141 MBytes/s, ETA 0s
Transferred:      216.964G / 216.964 GBytes, 100%, 97.925 MBytes/s, ETA 0s
Transferred:      278.285G / 278.285 GBytes, 100%, 94.010 MBytes/s, ETA 0s
Transferred:      133.379G / 133.379 GBytes, 100%, 93.928 MBytes/s, ETA 0s
Transferred:       16.019G / 16.019 GBytes, 100%, 88.026 MBytes/s, ETA 0s
Transferred:      228.938G / 228.938 GBytes, 100%, 95.987 MBytes/s, ETA 0s
Transferred:      131.325G / 131.325 GBytes, 100%, 94.779 MBytes/s, ETA 0s
Transferred:       23.407G / 23.407 GBytes, 100%, 95.572 MBytes/s, ETA 0s
Transferred:      159.429G / 159.429 GBytes, 100%, 96.870 MBytes/s, ETA 0s
Transferred:       41.109G / 41.109 GBytes, 100%, 94.574 MBytes/s, ETA 0s
Transferred:      124.701G / 124.701 GBytes, 100%, 90.346 MBytes/s, ETA 0s
Transferred:       24.389G / 24.389 GBytes, 100%, 89.646 MBytes/s, ETA 0s
Transferred:      243.451G / 243.451 GBytes, 100%, 95.643 MBytes/s, ETA 0s
Transferred:       55.014G / 55.014 GBytes, 100%, 76.338 MBytes/s, ETA 0s
Transferred:      258.020G / 258.020 GBytes, 100%, 95.333 MBytes/s, ETA 0s
Transferred:      124.378G / 124.378 GBytes, 100%, 94.727 MBytes/s, ETA 0s
Transferred:       40.197G / 40.197 GBytes, 100%, 95.773 MBytes/s, ETA 0s
Transferred:       87.790G / 87.790 GBytes, 100%, 93.060 MBytes/s, ETA 0s
Transferred:      172.436G / 172.436 GBytes, 100%, 95.953 MBytes/s, ETA 0s
Transferred:       59.994G / 59.994 GBytes, 100%, 91.374 MBytes/s, ETA 0s
Transferred:       76.195G / 76.195 GBytes, 100%, 85.108 MBytes/s, ETA 0s
Transferred:       96.275G / 96.275 GBytes, 100%, 93.045 MBytes/s, ETA 0s
Transferred:       53.685G / 53.685 GBytes, 100%, 82.943 MBytes/s, ETA 0s
Transferred:       17.039G / 17.039 GBytes, 100%, 93.364 MBytes/s, ETA 0s
Transferred:      158.626G / 158.626 GBytes, 100%, 92.061 MBytes/s, ETA 0s
Transferred:      239.452G / 239.452 GBytes, 100%, 92.567 MBytes/s, ETA 0s
Transferred:       27.765G / 27.765 GBytes, 100%, 87.716 MBytes/s, ETA 0s
Transferred:       56.844G / 56.844 GBytes, 100%, 56.422 MBytes/s, ETA 0s
Transferred:       82.740G / 82.740 GBytes, 100%, 91.488 MBytes/s, ETA 0s
Transferred:       82.740G / 82.740 GBytes, 100%, 91.488 MBytes/s, ETA 0s
Transferred:       82.740G / 82.740 GBytes, 100%, 91.488 MBytes/s, ETA 0s
Transferred:      183.194G / 183.194 GBytes, 100%, 80.137 MBytes/s, ETA 0s
Transferred:      275.402G / 275.402 GBytes, 100%, 89.062 MBytes/s, ETA 0s
Transferred:      444.401G / 444.401 GBytes, 100%, 79.436 MBytes/s, ETA 0s
Transferred:      501.588G / 501.588 GBytes, 100%, 94.176 MBytes/s, ETA 0s
Transferred:      144.059G / 144.059 GBytes, 100%, 92.281 MBytes/s, ETA 0s
Transferred:      467.308G / 467.351 GBytes, 100%, 91.822 MBytes/s, ETA 0s
Transferred:      467.351G / 467.351 GBytes, 100%, 91.778 MBytes/s, ETA 0s
Transferred:       84.802G / 84.802 GBytes, 100%, 92.329 MBytes/s, ETA 0s
Transferred:       84.802G / 84.802 GBytes, 100%, 92.329 MBytes/s, ETA 0s
Transferred:       84.802G / 84.802 GBytes, 100%, 92.329 MBytes/s, ETA 0s
Transferred:       27.506G / 27.506 GBytes, 100%, 90.289 MBytes/s, ETA 0s
Transferred:      534.056G / 534.056 GBytes, 100%, 92.955 MBytes/s, ETA 0s
Transferred:      135.962G / 135.962 GBytes, 100%, 64.301 MBytes/s, ETA 0s
Transferred:       88.051G / 88.051 GBytes, 100%, 70.517 MBytes/s, ETA 0s
Transferred:      133.837G / 133.837 GBytes, 100%, 96.167 MBytes/s, ETA 0s
Transferred:      292.048G / 292.048 GBytes, 100%, 91.542 MBytes/s, ETA 0s
Transferred:      206.222G / 206.222 GBytes, 100%, 87.866 MBytes/s, ETA 0s
Transferred:       77.119G / 77.119 GBytes, 100%, 86.547 MBytes/s, ETA 0s
Transferred:       16.493G / 16.493 GBytes, 100%, 80.564 MBytes/s, ETA 0s
Transferred:      267.565G / 267.565 GBytes, 100%, 91.899 MBytes/s, ETA 0s
Transferred:      166.406G / 166.406 GBytes, 100%, 93.690 MBytes/s, ETA 0s
Transferred:      101.606G / 101.606 GBytes, 100%, 40.342 MBytes/s, ETA 0s
Transferred:      611.482G / 611.482 GBytes, 100%, 49.997 MBytes/s, ETA 0s
Transferred:      153.375G / 153.375 GBytes, 100%, 44.691 MBytes/s, ETA 0s
Transferred:        3.149G / 3.149 GBytes, 100%, 81.977 MBytes/s, ETA 0s
Transferred:       92.489G / 92.489 GBytes, 100%, 78.211 MBytes/s, ETA 0s
Transferred:      405.767G / 405.767 GBytes, 100%, 50.990 MBytes/s, ETA 0s
Transferred:       94.843G / 94.843 GBytes, 100%, 38.315 MBytes/s, ETA 0s
Transferred:       33.028G / 33.028 GBytes, 100%, 46.858 MBytes/s, ETA 0s
Transferred:       62.707G / 62.707 GBytes, 100%, 49.339 MBytes/s, ETA 0s
Transferred:      339.641G / 339.641 GBytes, 100%, 57.143 MBytes/s, ETA 0s
Transferred:      394.795G / 394.795 GBytes, 100%, 68.411 MBytes/s, ETA 0s
Transferred:      103.673G / 103.673 GBytes, 100%, 31.659 MBytes/s, ETA 0s
Transferred:       58.273G / 58.273 GBytes, 100%, 43.528 MBytes/s, ETA 0s
Transferred:       36.250G / 36.257 GBytes, 100%, 15.579 MBytes/s, ETA 0s
Transferred:       36.259G / 36.259 GBytes, 100%, 15.536 MBytes/s, ETA 0s
Transferred:      129.572G / 129.572 GBytes, 100%, 40.055 MBytes/s, ETA 0s
Transferred:       16.839G / 16.839 GBytes, 100%, 2.987 MBytes/s, ETA 0s
Transferred:       16.231G / 16.231 GBytes, 100%, 29.926 MBytes/s, ETA 0s
Transferred:      114.317G / 114.363 GBytes, 100%, 52.578 MBytes/s, ETA 0s
Transferred:      114.364G / 114.364 GBytes, 100%, 51.879 MBytes/s, ETA 0s

I do get some bad days in there where Google just does not play nice and I normally max my gigabit, but some days, Google doesn't give me good rates. I have a little over two years of data (713 days).

My copies/sync use the defaults for transfers/checkers and since I mainly move large files, I change to 1G drive-chunk-size.

You'd have to share a log for each and we can see the difference. You can check the Google API Console and see exactly what's going on and what generates more API hits if you can ensure you are measuring just the one "process" that you have going on as I have one API key setup for my copies.

An example of my copy last night and you can see the breakdown of the API hits.

If your sync is doing less API than your copy, something would be off and that should be investigated.

i can't recommend enough for everyone to learn more about chaos and randomness (hope i'm not repeating myself too much here) without trying to put order into it.

it should be taught from basic school alongside math and history until doctoral alongside philosophy 404 and programming logic.

that is, supposing school should even be so scholar like that...

now...

you really need to get out of google to test issues with google even if not for anything else, because it's easily possible to do so (unlike testing for other forms of life such as the carbon based ones we know, for instance).

your tests there seem to average out at around 85mb/s which is a rather high speed and looks stable enough to me. even the bumpiness is expected under any normal circumstance. no idea what other meaning to extract from them. but they say nothing about copy vs sync.

sorry, but i won't bother with my google api and hopefully i won't need to bother with testing as well. just let it go, i don't need help there. :rofl:

(i do need help on my other topic, though. have you seen it?)

on another note: more than testing outside google, i used google for about 20 years and i had been on gmail before since the beta. i also can't stress enough on how blind and stupid i was to trust google with organising my data. i'm talking about the very core thing that google have always promised: offering a good way to organise the world data. they're actually terrible at it. despite some good deeds now and then. just terrible and way more biased than almost everyone would like to believe. please, do yourself a favour and give an honest try outside their system.

Good luck as I can see we've taken a turn away from looking at the data and moved into a different direction.

1 Like

haha yes...

all the way to a dead end. :joy: