Add flag to exit immediately when --max-duration reached

timevortex · March 8, 2021, 4:28pm

Creating this feature request based on my experience with Google Photos as per Detecting incomplete file download from Google Photos

I think adding a flag along the lines of --exit-immediately-on-max-duration would be useful for when intermittent API problems is preventing a transfer in-progress from finishing (or starting correctly) as I've experienced with Google Photos API.

ncw · March 9, 2021, 9:59am

Rclone should exit immediately on --max-duration provided --cutoff-mode is set to hard.

If it doesn't this is likely a bug...

There is a bug report about this already

Does that look like your issue?

timevortex · March 9, 2021, 5:56pm

The documentation for --cutoff-mode says:

I read this to mean that it doesn't impact --max-duration. Either way I haven't modified the default here so it should be set to hard.

Seems like the same bug (or related), specially if if you are surprised by this behaviour. I wasn't sure what the normal or expected behaviour is for this flag.

Experiment points to unexpected behaviour

From Detecting incomplete file download from Google Photos - #8 by pparth

So this experiment has run now at least a couple of times and I'm finding that rclone does not exit even an hour after --max-duration is reached.

Command: timeout -v --preserve-status -k 1m 7200s rclone sync --transfers=10 --max-duration=58m --retries 1 --timeout 1m --fast-list --gphotos-include-archived --gphotos-start-year=1900 --log-level INFO --log-file=REDACTED.txt gphotos:album album

Will add logs in separate replies to this thread to avoid hitting the post character limit.

timevortex · March 9, 2021, 5:58pm

Log from run 1

1.1. Things worked as expected until 58 minutes (`--max-duration 58m` was specified)

2021/03/09 13:25:27 INFO  : REDACTED to 1 year/VID_20191130_083619.mp4: Copied (Rcat, new)
2021/03/09 13:26:01 INFO  : 
Transferred:   	   21.661G / 21.661 GBytes, 100%, 7.190 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          670 / 979, 68%
Elapsed time:      56m0.0s
Transferring:
 * REDACTED to 1…ID_20191129_184746.mp4:  0% /off, 793.055k/s, -
 * REDACTED to 1…ID_20191130_082442.mp4:  0% /off, 780.858k/s, -
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 794.238k/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 760.958k/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 773.528k/s, -
 * REDACTED to 1…ID_20191201_165607.mp4:  0% /off, 791.668k/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 802.403k/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 778.665k/s, -
 * REDACTED to 1…ID_20191202_175108.mp4:  0% /off, 806.375k/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 724.947k/s, -

2021/03/09 13:26:37 INFO  : REDACTED to 1 year/VID_20191129_184746.mp4: Copied (Rcat, new)
2021/03/09 13:26:46 INFO  : REDACTED to 1 year/VID_20191202_175108.mp4: Copied (Rcat, new)
2021/03/09 13:26:53 INFO  : REDACTED to 1 year/VID_20191203_121741.mp4: Copied (Rcat, new)
2021/03/09 13:26:59 INFO  : REDACTED to 1 year/VID_20191203_162703-ANIMATION.gif: Copied (Rcat, new)
2021/03/09 13:27:01 INFO  : 
Transferred:   	   22.112G / 22.112 GBytes, 100%, 7.200 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          678 / 983, 69%
Elapsed time:      57m0.0s
Transferring:
 * REDACTED to 1…ID_20191130_082442.mp4:  0% /off, 812.658k/s, -
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 822.961k/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 793.355k/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 797.576k/s, -
 * REDACTED to 1…ID_20191201_165607.mp4:  0% /off, 822.419k/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 828.493k/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 812.111k/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 792.583k/s, -
 * REDACTED to 1…ID_20191203_101216.mp4:  0% /off, 738.811k/s, -
 * REDACTED to 1…ID_20191203_162703.mp4: transferring

2021/03/09 13:27:05 INFO  : REDACTED to 1 year/VID_20191130_082442.mp4: Copied (Rcat, new)
2021/03/09 13:27:22 INFO  : REDACTED to 1 year/VID_20191203_101216.mp4: Copied (Rcat, new)
2021/03/09 13:27:33 INFO  : REDACTED to 1 year/VID_20191201_165607.mp4: Copied (Rcat, new)
2021/03/09 13:28:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 7.211 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:      58m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 840.893k/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 885.514k/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 883.331k/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 851.871k/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 833.267k/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 888.271k/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 848.255k/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 868.196k/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 809.322k/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 804.297k/s, -

1.2. Nothing seems to be happening between 58m and 1h19m (21 minutes)

2021/03/09 13:28:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 7.211 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:      58m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 840.893k/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 885.514k/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 883.331k/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 851.871k/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 833.267k/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 888.271k/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 848.255k/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 868.196k/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 809.322k/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 804.297k/s, -

2021/03/09 13:29:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 7.079 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:      59m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 18.919k/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 18.428k/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 18.382k/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 17.728k/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 18.761k/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 18.484k/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 19.072k/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 18.066k/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 16.842k/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 16.663k/s, -

2021/03/09 13:30:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.951 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h0m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 403/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 392/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 391/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 377/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 399/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 393/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 406/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 385/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 358/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 355/s, -

2021/03/09 13:31:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.828 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h1m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 7/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 8/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 7/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 7/s, -

2021/03/09 13:32:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.709 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h2m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:33:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.594 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h3m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:34:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.483 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h4m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:35:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.376 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h5m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:36:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.272 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h6m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:37:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.172 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h7m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:38:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 6.074 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h8m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:39:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.980 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:     1h9m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:40:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.888 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h10m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:41:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.800 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h11m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:42:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.714 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h12m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:43:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.630 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h13m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:44:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.549 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h14m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:45:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.470 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h15m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:46:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.394 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h16m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:47:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.319 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h17m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:48:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.247 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h18m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:49:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.176 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h19m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

timevortex · March 9, 2021, 5:59pm

Log from run 1 (continued)

1.3. Between 1h19m and 1h25m we see lots of "context deadline exceeded" with all partial files in progress being deleted

2021/03/09 13:49:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.176 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 986, 69%
Elapsed time:    1h19m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_083816.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:49:12 NOTICE: REDACTED to 1 year/VID_20191203_083816.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:49:12 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:50:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.108 MBytes/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 985, 69%
Elapsed time:    1h20m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:51:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 5.041 MBytes/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 985, 69%
Elapsed time:    1h21m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:52:01 INFO  : 
Transferred:   	   22.569G / 22.569 GBytes, 100%, 4.976 MBytes/s, ETA 0s
Errors:                 2 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 985, 69%
Elapsed time:    1h22m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191201_080631.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_083619.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191202_144348.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:52:13 NOTICE: REDACTED to 1 year/VID_20191202_144348.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:52:13 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:52:30 NOTICE: REDACTED to 1 year/VID_20191202_083619.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:52:30 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:52:47 NOTICE: REDACTED to 1 year/VID_20191201_080631.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:52:47 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:53:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.912 MBytes/s, ETA 0s
Errors:                 5 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 982, 70%
Elapsed time:    1h23m0.0s
Transferring:
 * REDACTED to 1…_20191130_184505_1.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191130_185456.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_074327.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_124609.mp4:  0% /off, 0/s, -

2021/03/09 13:53:06 NOTICE: REDACTED to 1 year/VID_20191130_185456.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:53:08 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:53:23 NOTICE: REDACTED to 1 year/VID_20191130_184505_1.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:53:24 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:53:38 NOTICE: REDACTED to 1 year/VID_20191204_074327.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:53:41 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:53:53 NOTICE: REDACTED to 1 year/VID_20191204_124609.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:53:53 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:54:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.850 MBytes/s, ETA 0s
Errors:                 9 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 978, 70%
Elapsed time:    1h24m0.0s
Transferring:
 * REDACTED to 1…ID_20191203_162703.mp4:  0% /off, 0/s, -
 * REDACTED to 1…ID_20191204_075008.mp4:  0% /off, 0/s, -

2021/03/09 13:54:09 NOTICE: REDACTED to 1 year/VID_20191204_075008.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:54:09 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:54:26 NOTICE: REDACTED to 1 year/VID_20191203_162703.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 13:54:26 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 13:55:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.790 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h25m0.0s

1.4. Between 1h25m and 2h, there are no pending transfers but rclone still does not exit

2021/03/09 13:55:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.790 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h25m0.0s

2021/03/09 13:56:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.731 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h26m0.0s

2021/03/09 13:57:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.674 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h27m0.0s

2021/03/09 13:58:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.618 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h28m0.0s

2021/03/09 13:59:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.563 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h29m0.0s

2021/03/09 14:00:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.510 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h30m0.0s

2021/03/09 14:01:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.458 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h31m0.0s

2021/03/09 14:02:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.407 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h32m0.0s

2021/03/09 14:03:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.357 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h33m0.0s

2021/03/09 14:04:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.308 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h34m0.0s

2021/03/09 14:05:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.260 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h35m0.0s

2021/03/09 14:06:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.214 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h36m0.0s

2021/03/09 14:07:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.168 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h37m0.0s

2021/03/09 14:08:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.123 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h38m0.0s

2021/03/09 14:09:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.080 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h39m0.0s

2021/03/09 14:10:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 4.037 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h40m0.0s

2021/03/09 14:11:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.995 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h41m0.0s

2021/03/09 14:12:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.954 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h42m0.0s

2021/03/09 14:13:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.914 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h43m0.0s

2021/03/09 14:14:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.875 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h44m0.0s

2021/03/09 14:15:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.836 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h45m0.0s

2021/03/09 14:16:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.798 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h46m0.0s

2021/03/09 14:17:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.761 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h47m0.0s

2021/03/09 14:18:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.725 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h48m0.0s

2021/03/09 14:19:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.689 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h49m0.0s

2021/03/09 14:20:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.654 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h50m0.0s

2021/03/09 14:21:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.620 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h51m0.0s

2021/03/09 14:22:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.586 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h52m0.0s

2021/03/09 14:23:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.553 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h53m0.0s

2021/03/09 14:24:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.521 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h54m0.0s

2021/03/09 14:25:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.489 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h55m0.0s

2021/03/09 14:26:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.457 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h56m0.0s

2021/03/09 14:27:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.427 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h57m0.0s

2021/03/09 14:28:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.396 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h58m0.0s

2021/03/09 14:29:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.367 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:    1h59m0.0s

timeout: sending signal TERM to command ‘rclone’
2021/03/09 14:30:01 INFO  : 
Transferred:   	   22.570G / 22.570 GBytes, 100%, 3.337 MBytes/s, ETA 0s
Errors:                11 (retrying may help)
Checks:             81798 / 81798, 100%
Transferred:          684 / 976, 70%
Elapsed time:     2h0m0.0s

timevortex · March 9, 2021, 6:00pm

Log from run 2

2.1. Things worked as expected until 58 minutes (`--max-duration 58m` was specified)

Transferred:       21.350G / 21.350 GBytes, 100%, 6.511 MBytes/s, ETA 0s
Transferred:        16064 / 26081, 62%
Elapsed time:      57m0.0s
Transferring:
 * REDACTED on…20210131_184213619.mp4:  0% /off, 776.702k/s, -
 * REDACTED on…20210131_185810685.mp4:  0% /off, 775.538k/s, -
 * REDACTED on…20210202_080518643.mp4:  0% /off, 778.032k/s, -
 * REDACTED on…20210202_144624586.mp4:  0% /off, 810.301k/s, -
 * REDACTED on…20210203_111355160.mp4:  0% /off, 795.413k/s, -
 * REDACTED on…20210203_151246579.mp4:  0% /off, 778.378k/s, -
 * REDACTED on…20210203_160816659.mp4:  0% /off, 584.560k/s, -
 * REDACTED on…20210204_103732593.jpg: transferring
 * REDACTED on…20210204_103005823.jpg:  0% /off, 1.996M/s, -
 * REDACTED on…20210204_103736830.jpg: transferring

2021/03/09 15:27:02 INFO  : REDACTED onwards/PXL_20210204_103005823.jpg: Copied (Rcat, new)
2021/03/09 15:27:02 INFO  : REDACTED onwards/PXL_20210203_111355160.mp4: Copied (Rcat, new)
2021/03/09 15:27:03 INFO  : REDACTED onwards/PXL_20210204_103732593.jpg: Copied (Rcat, new)
2021/03/09 15:27:03 INFO  : REDACTED onwards/PXL_20210202_144624586.mp4: Copied (Rcat, new)
2021/03/09 15:27:03 INFO  : REDACTED onwards/PXL_20210204_103736830.jpg: Copied (Rcat, new)
2021/03/09 15:27:05 INFO  : REDACTED onwards/PXL_20210204_114024497.jpg: Copied (Rcat, new)
2021/03/09 15:27:06 INFO  : REDACTED onwards/PXL_20210204_114454878.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:06 INFO  : REDACTED onwards/PXL_20210131_184213619.mp4: Copied (Rcat, new)
2021/03/09 15:27:06 INFO  : REDACTED onwards/PXL_20210204_114550294.jpg: Copied (Rcat, new)
2021/03/09 15:27:07 INFO  : REDACTED onwards/PXL_20210204_114556266.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:08 INFO  : REDACTED onwards/PXL_20210204_114619044.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:09 INFO  : REDACTED onwards/PXL_20210204_114620207.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:09 INFO  : REDACTED onwards/PXL_20210204_152312340.jpg: Copied (Rcat, new)
2021/03/09 15:27:11 INFO  : REDACTED onwards/PXL_20210204_155609547.jpg: Copied (Rcat, new)
2021/03/09 15:27:12 INFO  : REDACTED onwards/PXL_20210204_163655903.jpg: Copied (Rcat, new)
2021/03/09 15:27:13 INFO  : REDACTED onwards/PXL_20210204_164408424.jpg: Copied (Rcat, new)
2021/03/09 15:27:14 INFO  : REDACTED onwards/PXL_20210205_074953083.jpg: Copied (Rcat, new)
2021/03/09 15:27:14 INFO  : REDACTED onwards/PXL_20210205_090552041.jpg: Copied (Rcat, new)
2021/03/09 15:27:15 INFO  : REDACTED onwards/PXL_20210205_083421518.jpg: Copied (Rcat, new)
2021/03/09 15:27:16 INFO  : REDACTED onwards/PXL_20210205_105148869.jpg: Copied (Rcat, new)
2021/03/09 15:27:17 INFO  : REDACTED onwards/PXL_20210205_105154268.jpg: Copied (Rcat, new)
2021/03/09 15:27:18 INFO  : REDACTED onwards/PXL_20210205_105209585.jpg: Copied (Rcat, new)
2021/03/09 15:27:19 INFO  : REDACTED onwards/PXL_20210205_180142829.jpg: Copied (Rcat, new)
2021/03/09 15:27:20 INFO  : REDACTED onwards/PXL_20210205_180147238.jpg: Copied (Rcat, new)
2021/03/09 15:27:21 INFO  : REDACTED onwards/PXL_20210205_191431777.jpg: Copied (Rcat, new)
2021/03/09 15:27:23 INFO  : REDACTED onwards/PXL_20210206_114009433.jpg: Copied (Rcat, new)
2021/03/09 15:27:23 INFO  : REDACTED onwards/PXL_20210206_114001389-ANIMATION.gif: Copied (Rcat, new)
2021/03/09 15:27:24 INFO  : REDACTED onwards/PXL_20210206_114013991.jpg: Copied (Rcat, new)
2021/03/09 15:27:24 INFO  : REDACTED onwards/PXL_20210204_145531949.mp4: Copied (Rcat, new)
2021/03/09 15:27:24 INFO  : REDACTED onwards/PXL_20210206_114015986.jpg: Copied (Rcat, new)
2021/03/09 15:27:25 INFO  : REDACTED onwards/PXL_20210206_114016909.jpg: Copied (Rcat, new)
2021/03/09 15:27:25 INFO  : REDACTED onwards/PXL_20210206_114029954.jpg: Copied (Rcat, new)
2021/03/09 15:27:27 INFO  : REDACTED onwards/PXL_20210206_152305323.jpg: Copied (Rcat, new)
2021/03/09 15:27:27 INFO  : REDACTED onwards/PXL_20210206_183929031.jpg: Copied (Rcat, new)
2021/03/09 15:27:28 INFO  : REDACTED onwards/PXL_20210206_152047304.jpg: Copied (Rcat, new)
2021/03/09 15:27:29 INFO  : REDACTED onwards/PXL_20210206_183931045.jpg: Copied (Rcat, new)
2021/03/09 15:27:29 INFO  : REDACTED onwards/PXL_20210207_102756700.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:30 INFO  : REDACTED onwards/PXL_20210207_141541107.jpg: Copied (Rcat, new)
2021/03/09 15:27:30 INFO  : REDACTED onwards/PXL_20210207_143804711.jpg: Copied (Rcat, new)
2021/03/09 15:27:31 INFO  : REDACTED onwards/PXL_20210207_143808470.jpg: Copied (Rcat, new)
2021/03/09 15:27:32 INFO  : REDACTED onwards/PXL_20210207_144934129.jpg: Copied (Rcat, new)
2021/03/09 15:27:34 INFO  : REDACTED onwards/PXL_20210207_181021447.jpg: Copied (Rcat, new)
2021/03/09 15:27:34 INFO  : REDACTED onwards/PXL_20210207_181101119.jpg: Copied (Rcat, new)
2021/03/09 15:27:35 INFO  : REDACTED onwards/PXL_20210204_145754248.mp4: Copied (Rcat, new)
2021/03/09 15:27:37 INFO  : REDACTED onwards/PXL_20210207_182505302.jpg: Copied (Rcat, new)
2021/03/09 15:27:38 INFO  : REDACTED onwards/PXL_20210208_082515001.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:39 INFO  : REDACTED onwards/PXL_20210206_114001389.mp4: Copied (Rcat, new)
2021/03/09 15:27:39 INFO  : REDACTED onwards/PXL_20210208_105356831.jpg: Copied (Rcat, new)
2021/03/09 15:27:41 INFO  : REDACTED onwards/PXL_20210208_105619871.jpg: Copied (Rcat, new)
2021/03/09 15:27:41 INFO  : REDACTED onwards/PXL_20210208_105623001.jpg: Copied (Rcat, new)
2021/03/09 15:27:42 INFO  : REDACTED onwards/PXL_20210131_185810685.mp4: Copied (Rcat, new)
2021/03/09 15:27:43 INFO  : REDACTED onwards/PXL_20210208_160926173.jpg: Copied (Rcat, new)
2021/03/09 15:27:44 INFO  : REDACTED onwards/PXL_20210208_160938261.jpg: Copied (Rcat, new)
2021/03/09 15:27:45 INFO  : REDACTED onwards/PXL_20210208_160946686.jpg: Copied (Rcat, new)
2021/03/09 15:27:45 INFO  : REDACTED onwards/PXL_20210203_151246579.mp4: Copied (Rcat, new)
2021/03/09 15:27:46 INFO  : REDACTED onwards/PXL_20210208_162605867.jpg: Copied (Rcat, new)
2021/03/09 15:27:47 INFO  : REDACTED onwards/PXL_20210208_163104122.jpg: Copied (Rcat, new)
2021/03/09 15:27:48 INFO  : REDACTED onwards/PXL_20210208_163108608.jpg: Copied (Rcat, new)
2021/03/09 15:27:48 INFO  : REDACTED onwards/PXL_20210208_105403063.mp4: Copied (Rcat, new)
2021/03/09 15:27:48 INFO  : REDACTED onwards/PXL_20210208_182618885.jpg: Copied (Rcat, new)
2021/03/09 15:27:49 INFO  : REDACTED onwards/PXL_20210209_072447689.jpg: Copied (Rcat, new)
2021/03/09 15:27:50 INFO  : REDACTED onwards/PXL_20210209_072501046.jpg: Copied (Rcat, new)
2021/03/09 15:27:51 INFO  : REDACTED onwards/PXL_20210209_072523994.jpg: Copied (Rcat, new)
2021/03/09 15:27:52 INFO  : REDACTED onwards/PXL_20210209_152426934.jpg: Copied (Rcat, new)
2021/03/09 15:27:52 INFO  : REDACTED onwards/PXL_20210209_152944780.jpg: Copied (Rcat, new)
2021/03/09 15:27:53 INFO  : REDACTED onwards/PXL_20210207_181043185.mp4: Copied (Rcat, new)
2021/03/09 15:27:53 INFO  : REDACTED onwards/PXL_20210209_154811726.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:55 INFO  : REDACTED onwards/PXL_20210209_154827630.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:56 INFO  : REDACTED onwards/PXL_20210209_163036164.PORTRAIT.jpg: Copied (Rcat, new)
2021/03/09 15:27:57 INFO  : REDACTED onwards/PXL_20210209_163125731.jpg: Copied (Rcat, new)
2021/03/09 15:27:59 INFO  : REDACTED onwards/PXL_20210209_163135627.jpg: Copied (Rcat, new)
2021/03/09 15:28:00 INFO  : REDACTED onwards/PXL_20210203_160816659.mp4: Copied (Rcat, new)
2021/03/09 15:28:01 INFO  : REDACTED onwards/PXL_20210209_163711172.jpg: Copied (Rcat, new)
2021/03/09 15:28:02 INFO  :
Transferred:       21.769G / 21.769 GBytes, 100%, 6.522 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:      58m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 1.204M/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 1.138M/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 1.300M/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 509.969k/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 509.961k/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 251.972k/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 507.937k/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

timevortex · March 9, 2021, 6:01pm

Log from run 2 (continued)

2.2. Nothing seems to be happening between 58m and 1h30 minutes (32 minutes)

Transferred:   	   21.769G / 21.769 GBytes, 100%, 6.522 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:      58m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 1.204M/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 1.138M/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 1.300M/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 509.969k/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 509.961k/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 251.972k/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 507.937k/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:29:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 6.410 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:      59m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 25.666k/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 25.675k/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 25.494k/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 6.562k/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 6.562k/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 3.492k/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 3.492k/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:30:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 6.301 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h0m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 546/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 547/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 543/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 139/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 139/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 74/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 74/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:31:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 6.196 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h1m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 11/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 11/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 11/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 2/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 2/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 1/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 1/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:32:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 6.094 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h2m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:33:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.996 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h3m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:34:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.901 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h4m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:35:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.809 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h5m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:36:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.719 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h6m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:37:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.632 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h7m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:38:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.548 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h8m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:39:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.467 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:     1h9m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:40:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.387 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h10m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:41:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.310 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h11m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:42:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.236 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h12m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:43:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.163 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h13m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:44:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.092 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h14m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:45:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 5.023 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h15m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:46:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.956 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h16m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:47:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.891 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h17m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:48:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.827 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h18m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:49:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.765 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h19m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:50:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.705 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h20m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:51:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.646 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h21m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:52:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.589 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h22m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:53:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.533 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h23m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:54:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.478 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h24m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:55:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.425 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h25m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:56:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.373 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h26m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:57:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.322 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h27m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:58:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.272 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h28m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 15:59:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.224 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h29m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:00:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.176 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h30m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

timevortex · March 9, 2021, 6:04pm

Log from run 2 (continued)

2.3. Between 1h30m and 1h34m we see lots of "context deadline exceeded" with some partial file deletions as well as API ERROR for listings

2021/03/09 16:00:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.176 MBytes/s, ETA 0s
Transferred:        16210 / 26228, 62%
Elapsed time:    1h30m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155509353.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:00:03 NOTICE: REDACTED onwards/PXL_20210209_155509353.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:00:03 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:00:20 ERROR : REDACTED_ALBUM_NAME1: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/09 16:00:36 ERROR : REDACTED_ALBUM_NAME2: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/09 16:00:53 ERROR : REDACTED_ALBUM_NAME3: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/09 16:01:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.130 MBytes/s, ETA 0s
Errors:                 4 (retrying may help)
Transferred:        16210 / 26227, 62%
Elapsed time:    1h31m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_170622523.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_154841544.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:01:10 NOTICE: REDACTED onwards/PXL_20210208_170622523.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:01:10 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:01:43 NOTICE: REDACTED onwards/PXL_20210209_154841544.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:01:43 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:02:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.084 MBytes/s, ETA 0s
Errors:                 6 (retrying may help)
Transferred:        16210 / 26225, 62%
Elapsed time:    1h32m0.0s
Transferring:
 * REDACTED on…20210202_080518643.mp4:  0% /off, 0/s, -
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…210207_143752192~2.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_155155116.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:02:16 NOTICE: REDACTED onwards/PXL_20210207_143752192~2.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:02:16 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:02:33 NOTICE: REDACTED onwards/PXL_20210202_080518643.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:02:33 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:02:49 NOTICE: REDACTED onwards/PXL_20210209_155155116.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:02:49 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:03:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 4.040 MBytes/s, ETA 0s
Errors:                 9 (retrying may help)
Transferred:        16210 / 26222, 62%
Elapsed time:    1h33m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163041688.mp4:  0% /off, 0/s, -
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:03:22 NOTICE: REDACTED onwards/PXL_20210209_163041688.mp4: Removing partially written file on error: context deadline exceeded
2021/03/09 16:03:22 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	github.com/rclone/rclone/fs/operations/reopen.go:28
runtime.doInit
	runtime/proc.go:5652
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.doInit
	runtime/proc.go:5647
runtime.main
	runtime/proc.go:191
runtime.goexit
	runtime/asm_amd64.s:1374
2021/03/09 16:04:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.997 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h34m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2.4. From 1h34m until 2h, three transfers seem to have hung indefinitely until `timeout` kills rclone

2021/03/09 16:04:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.997 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h34m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:05:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.954 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h35m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:06:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.912 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h36m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:07:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.872 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h37m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:08:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.832 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h38m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:09:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.793 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h39m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:10:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.754 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h40m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:11:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.717 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h41m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:12:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.680 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h42m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:13:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.644 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h43m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:14:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.608 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h44m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:15:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.574 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h45m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:16:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.540 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h46m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:17:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.506 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h47m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:18:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.473 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h48m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:19:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.441 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h49m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:20:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.410 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h50m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:21:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.379 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h51m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:22:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.348 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h52m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:23:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.318 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h53m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:24:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.289 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h54m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:25:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.260 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h55m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:26:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.232 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h56m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:27:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.204 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h57m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:28:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.177 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h58m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

2021/03/09 16:29:02 INFO  : 
Transferred:   	   21.769G / 21.769 GBytes, 100%, 3.150 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Transferred:        16210 / 26221, 62%
Elapsed time:    1h59m0.0s
Transferring:
 * REDACTED on…20210208_082915459.mp4: transferring
 * REDACTED on…20210209_163602964.mp4: transferring
 * REDACTED on…20210209_163931600.mp4: transferring

timeout: sending signal TERM to command ‘rclone’

Observations based on run 1 & 2

The "context deadline exceed" doesn't seem to be the same amount at each run.
In run 1 we see rclone not exiting even though it is not transferring anything. Can't see what it's hung on.
In run 2 we see that rclone never clears 3 files that it hadn't even started to transfer when --max-timeout was reached.

timevortex · March 9, 2021, 6:10pm

@ncw If these logs and experiments are helpful to finding where the bug or problem might be then I'm happy to keep adding more since I'm probably going to be running this for a few more weeks still (I haven't even started runnng sync on shared-album yet).

But if you already know where the bug is then I'll stop spending time sharing these logs

Also please let me know if there is anything you'd like me to try in my experiments!

ncw · March 10, 2021, 8:34pm

Thanks for your logs.

What would be helpful is if you kill -QUIT rclone when it is hung and collect the backtrace which will show what rclone is doing.

Or alternatively if you want to not be quite so vigorous then add the --rc flag and collect this log curl http://localhost:5572/debug/pprof/goroutine?debug=1 this will show all of rclone's goroutines and what they are doing.

It will be quite long, so one log when it is making no progress on the files after the 1 hour has elapsed would be great and if you can get one a log when it appeared to be transferring no files, but wasn't ending.

Then I can see if there is something specific about the google photos backend which is causing this.

timevortex · March 10, 2021, 9:11pm

This will be easy since timeout allows specifying the signal to send (by default it sends TERM). I've just set this for next run:
timeout -v -s QUIT --preserve-status -k 1m 175m rclone [...]

I decided to extend timeout from 2 hours to 2 hours 55 minutes (175m) to see if the extra time somehow allows rclone to finish neatly. This is the max I can wait before rclone needs to be killed because I can't do longer than 3 hours due to the way I have the cronjobs scheduled.

I have also changed --log-level INFO to --log-level DEBUG to help see more info. Will be very annoying to shift through the log file but maybe it'll provide the necessary insights.

I've reverted my rclone version to back to stable (v1.54.1) so there is no confusion when I report back (I have been experimenting with my own changes as per https://github.com/rclone/rclone/issues/4504#issuecomment-795526121)

Question

Should I also add --rc as per your suggestion? Does it include anything that's not already covered by the above changes?

Sounds like --rc is for debugging interactively and I can't guarantee that I'll be around the next time the cronjob kicks off.

timevortex · March 11, 2021, 5:08am

I've got a new log to share with debug + backtrace.

Command run: timeout -v -s QUIT --preserve-status -k 1m 175m rclone sync --transfers=10 --max-duration=59m --retries 1 --timeout 1m --fast-list --gphotos-include-archived --gphotos-start-year=1900 --log-level DEBUG --log-file=REDACTED.txt gphotos:shared-album shared-album

I've uploaded logs from 59m until the QUIT gets called by timeout:

Pacer retrying non-stop seems to be a big thing that stands out to me.

Hope this helps!

I'll turn off verbose mode now for my next run (happy to leave the backtrace in as it doesn't clutter up the logs).

timevortex · March 11, 2021, 5:48am

Here is another run which did finish by itself. In this case I had run out of API quota starting at 5m:

It's still not great that it took over 1 hour for rclone to finally give up and exit even though API errors started at 5m into the run!

ncw · March 11, 2021, 2:50pm

That is an alternative way of doing it.

I analysed your backtrace - condensed it looks like this

1: semacquire [87 minutes]
    sync         sema.go:56            runtime_Semacquire(#17)
    sync         waitgroup.go:130      (*WaitGroup).Wait(#16)
    sync         sync.go:430           (*syncCopyMove).stopTransfers(#15)
    sync         sync.go:865           (*syncCopyMove).run(#15, #21, #11)
    sync         sync.go:1078          runSyncCopyMove(#7, #21, #11, #28, #9, #38, 0x3, #29, 0x4428ae)
    sync         sync.go:1084          Sync(#7, #21, #11, #28, #9, #38, #2, #3, #45)
    sync         sync.go:55            glob..func1.1(#45, #21)
    cmd          cmd.go:250            Run(#18, #13, #55)
    sync         sync.go:53            glob..func1(#13, #20, 0x2, 0xe)
    cobra        command.go:854        (*Command).execute(#13, #19, 0xe, 0xe, #13, #19)
    cobra        command.go:958        (*Command).ExecuteC(#12, 0, 0x406325, #24)
    cobra        command.go:895        (*Command).Execute(...)
    cmd          cmd.go:543            Main()
    main         rclone.go:14          main()
10: chan receive [0~3 minutes] [Created by sync.(*syncCopyMove).startTransfers @ sync.go:422]
    pacer        pacer.go:159          (*Pacer).beginCall(#30)
    pacer        pacer.go:196          (*Pacer).call(#30, *, 0xa, 0x40, #4)
    pacer        pacer.go:216          (*Pacer).Call(#30, *, *, *)
    googlephotos googlephotos.go:939   (*Object).Open(*, #8, #31, *, 0x1, 0x1, 0, 0, *, 0x80)
    operations   reopen.go:96          (*ReOpen).open(*, *, *)
    operations   reopen.go:47          NewReOpen(#8, #31, #10, *, 0xa, *, 0x1, 0x1, 0, 0, ...)
    operations   operations.go:440     Copy(#8, #31, #11, #28, 0, 0, *, *, #10, *, ...)
    sync         sync.go:395           (*syncCopyMove).pairCopyOrMove(#15, #8, #31, #27, #11, #28, *, #16)
1: sleep [Created by pacer.(*Pacer).beginCall @ pacer.go:166]
    time         time.go:188           Sleep(#14)
    pacer        pacer.go:167          (*Pacer).beginCall.func1(#30, #14)
1: select [Created by cmd.StartStats @ cmd.go:355]
    cmd          cmd.go:359            StartStats.func2(#39, #25)
1: select [Created by view.init.0 @ worker.go:34]
    view         worker.go:276         (*worker).start(#26)

You can see the 10 transfer go routines and they are all running Copy which is in turn calling NewReOpen to reopen a failed transfer.

The transfers are all failing with context deadline exceeded because the max duration has been exceeded, however they are being retried because that is a retriable error. These retries are taking a very long time though since the time between retries has gone up to 16 seconds. I think also that these might be retried 10 times also so each failing thing is being retried 100 times at 16 seconds is 26 minutes.

So I think what is happening is

the context deadline exceeded errors are being retried when they shouldn't be
NewOpen is doing --low-level-retries and also so is Copy so it is doing the number of retries squared...
the time between retries is 16 seconds (that is according to google's spec)

I've attempted to fix the first of these which should fix all the problems - can you give this a go

v1.55.0-beta.5251.85cc93c20.fix-context-deadline on branch fix-context-deadline (uploaded in 15-30 mins)

If that works I'll put that change into all the backends...

timevortex · March 11, 2021, 4:00pm

Built & deployed for my next scheduled run (in ~1.5 hours). Will report back with success or backtraces if timeout ends up calling QUIT on it.

timevortex · March 11, 2021, 6:59pm

Success!

Relevant log bits below as FYI. I ran this at log level DEBUG but for the below bits I'm only showing egrep -v DEBUG as the DEBUG messages don't seem relevant. If you want to see them please let me know and I can upload them to Google Drive again.

Starting info:
2021/03/11 18:30:01 DEBUG : rclone: Version "v1.55.0-beta.5251.85cc93c20.fix-context-deadline" starting with parameters ["rclone" "sync" "--transfers=10" "--max-duration=59m" "--retries" "1" "--timeout" "1m" "--fast-list" "--gphotos-include-archived" "--gphotos-start-year=1900" "--log-level" "DEBUG" "--log-file=REDACTED.txt" "gphotos:shared-album" "shared-album"]

Also: 2021/03/11 18:30:01 INFO : Local file system at /REDACTED/shared-album: Transfer session deadline: 2021/03/11 19:29:01

Logs from 59m until end:

2021/03/11 19:29:01 INFO  : 
Transferred:   	   24.687G / 24.687 GBytes, 100%, 7.239 MBytes/s, ETA 0s
Transferred:        18506 / 28524, 65%
Elapsed time:      59m0.0s
Transferring:
 *  REDACTED_Trip/IMG_20181119_123240.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134444.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134450.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134500.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134507.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134816.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134822.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_134828.jpg: transferring
 *  REDACTED_Trip/IMG_20181119_123713.jpg:  0% /off, 0/s, -
 *  REDACTED_Trip/IMG_20181119_134836.jpg: transferring

2021/03/11 19:29:01 ERROR : REDACTED_ALBUM_NAME_1: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/11 19:29:01 ERROR : REDACTED_ALBUM_NAME_2: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/11 19:29:01 ERROR : REDACTED_ALBUM_NAME_3: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/11 19:29:01 ERROR : REDACTED_ALBUM_NAME_4: error reading source directory: couldn't list files: Post "https://photoslibrary.googleapis.com/v1/mediaItems:search": context deadline exceeded
2021/03/11 19:29:01 NOTICE: REDACTED_Trip/IMG_20181119_123713.jpg: Removing partially written file on error: context deadline exceeded
2021/03/11 19:29:01 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134507.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_Z_WzuDb6LDHP8KHrl3Fmk1zVPxxSLdouc3Gy9OQjPegrKgMqpX7-zAi5vMxFzPCcC0g17a7PlPyQs805qGj1PdHj-uDh4AwnM0o4T65g4dsh3pIOMhGrMFuzT93Qm_VYeBSHuF-ZXbH1592uiij_bZyKD_irnIAAQZ9u9EJcXmcMdz_SLbDTT0auslj-bxrgHeOuvFYvEOt20h_i1hL1tgMz3SOr18PWLWNUAxPA8emFg88WJaEuf1PdbYTHZPyYZfSiAZOZxwb4-jMsuoYRXc72s_zev-yyXm8LzbWnDskS8yw4UzqHn3EscnB7dadF9yvTHh-LyxUf5lsEKZ4ZsXu8cgJfP7EF_dGcljPLZHi0KAESy3isbzhd4IgDE8LTfGFYaMniWcJ6B1-vwLMtAb7oIPYsE7Cd6rx0WgQGQQ1FKSoIyQmwkiHaMwaUwYVRM8F2q645ZPB0h8J7hqiDtAVdXUubuvTXCO4TLJQjNLGeP6yKWDfkMd8eyYv8foyu20nfoKWhyJmXuO0_bGTYKDy0zISN6NPkddH1FCCqd9sR5IsQMfqdFYWHlGN4rqsK7dITn8uDP4H1qYY2UD96RQapK_c7VL7pSeKf9IdDKW3INVys-28qCS6nKsV2gcebUBp7Vm34zTL1vQpL4tdG6Zw0diJhnC8f5ik_T6se-dit5vm_GkJejM2e3mk8mB44ov6XrEIc0DbTfrzt-YXFZ9Z4TN809u_u5TRaM8Ohw_6c__qyocd4QPPEGGWjdDZHePMHg0KrXdLYcjO8w4JNtRyMYYSxJUaUxc6rLD0uMT9WfbI-Ov04fvUYUHLEDTdepR5dFdo1R0w3lk_w=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134500.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_bWYpbbBjxaEVCUySybgQmRbOUQc0bm0C7dGQMwuemmjfhHYLp0LXB3Wm98KvUayxD9GWbW_w8bwu7xccaBpBWcMRbqP0BoAzAiWQIXNYN6QsyTGnin2WSsseAcgf_0YbbgUtRBWgVXRhHCxegdHM7IIte9p47RSFclTHoy9Jq9ihg9Rv-Iz6SfkqogcbwH785SHFhpYMYxz8bMtPm7Jxk8G8uUdwVrIXY4q153AmkA9Mj5lai1BkhW3lRQUHj5jivB1q0yWWIOyorsLX1tlbpP0W4porwlGvJ-Gg9cPLHnLzvKddEH4rMCVk8UtsTfioFfNCCFgHK5H1LE6M6dwZsvXWjOkkQU3gVyjT_QVcO3HPpkJ_dYAn0Jtf-knj7C8c-RuREqHEaOspdNSKsKHEQtvXPHi7PyYleAXXXErFfrCjPm3W8PlWkfI3XuMZRxJc-tyFHbCwVSaWCNrkSokmATCjCJ8D5XHgjndty_AuZa6v3LUxeEx5jn2vxfY_kuBOjzBpSE3tMDDk3BjDoR625lKQWFikMJeAcwJQaTx12uv-l1MHDjbxLzL2D5cqWn2fZihT26yWA1xFDSePduAbKZXWQF0p3skCAM8vJeR9v7V4PCtA_NvdYmWXdGKn2Q96T76uDWhOHP-8mORieL6ww_YTr0Bl-Y22j8vDWaC2qvxOLaACjz3HbOgqBMtPovP6SHi65WCE1Tq3L4AcQnUUziaueH0pgXUQUc8z14sHs_W7bQWR3UOnG5ZcnDEQ52NSrlpT1BLRFHWeYR6FY1nMkV7gfCI9mMFXn8YFSxaFeFi3kFuXtIByr3FZW6od35gEKOWDcdaynOOAX2rA=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134450.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_bwjnhIgOjoKH7L9TzbYazjCPEDcbw8W9FyW7K68Xa61scPJ9pTmd0dfaloJsKHCoL-KSDarMB-0GIbnjUjaoU1A6S-PVUNdKf7sXiOOAUNhBpw_wKoQbjY98C3FktsVEiPDL46R2AAvcIUNtPJ30QdhkdaRRj6_uz0z_CsVoptC5uj9LAEN2W4wMAcPiFuGFO1-Lgtqtg6wCFRotrR28ww3zNosPMI78sFF1DTHMCsTeu9xVwGXtryxdyx9UAwxI2AzhKu-wzoiv5N_yJrkdJsbDnX3jk4U9YqWmaHkZZg-iP5nXGFhu0QgjTONmA9X8yoZwCoS-2eiqUQoMwzED7_FPaGCv6345rlZSPj1pearjZfW50B_gXOErEtCmH30aHvC4H5_lujkHdGRhlEwqu3YqPjS5kSN5VczSN4lnCFIAW62u7-iAkDAieKOvNHI5uUSnL4xbUevXi8S7bZABYGs587jhHiWvIEKnjLT3hHxOFiFkPMcg7xIrJn1f7q9wzXszxAr6RWF1QhAe4ZTI8y9PHr--Mlu4HDJwAD0cVBXksGSVzrgCQYmoA2PePGmQv-CmpiM6VmR89vSywfjUzhu-lUUv2AcR1UPGAMxWU4RdOhno2mIH7EheCN7x5TISqatmvSFS0Ddx2SQSwpHlc1a8G9CdqkzP4dVLWlGfKrsPghE-I4Qh4KeTnXQ3WoQYgQlJ0sBUZ6SpqOi7jT04HbZWzx3bXnGZ5uJufK0Mjx5rfboV0nam-0a797nYzhmNHymr728NnHShOKWYF7n4bkaJt2Fyv6uEEyzmd3w7Vh6_nJ4n8YptUoND8ZaUrQ6tfiKN5AvHfW6x9BLQ=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134444.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_Z1GUUEQDBkl5bcPWs_rCF2Mda2CdrAPs9tgIyyXxv4Zc_GPtrJO42eqX9YQgUB_vAxayIAAV6yozDYbPxpZomHqqdhoAsL5gS6PItHD1PBJhqPFJMpJXcLaw-BVX6eLaGwnPyj9G5tMMOOgy1E88h9ggmjJqDXIzIgZnreVWs-e2YmR3r4muHGmV7jmdsFDnVi4HY1_FctA9wBw4KkmmqdzTtZGpo4POJ4VLeUV5tgXq_3vUBTdj69XUgX70qgrjdUEBZlxNF2SnVIi7ZE4ODGah3VtGENVTYopEJlLiAL0kCGE9LNNItUv1ECkSc787kdAaxTi3UEHmaIFh05rGuXN6HLYBCkcgZ43FFTlw77HPzvSLO2CLGlZGMIOqT9XsD841GuGWjm6RH5WMHrGG5afvM0RsjrjceZ60o6R8ak8ue5EJy40PVjXq77osfwJC-nzvgQOGj9B8wwXQUM93JhruUpIxx355EzwpMQeA5at_hY2K04GdgXpQgcrUhArWU_eI1jw4MkpotyBqusk-N0gQ210I5pY9BHbKiVEFOjjYadCVYaogzJH2gjte7ppNC_dkd8V3teoV6fGcrdWnUhQDmpKUQXlWxH9y-31J4BYwMhAVD3Z4d4RiK-Fo3q82ojhDOrRmN5Ch2vIRnwM578Y4jxWvFt8P_A_FFG5pdL7DBRMA7fJA39O2kEKH9u--bSYVpA1gBHVZWV0OxjrjHl9bcuwuOe2hM8xYDSLzO-uVdykzAyMwpHJkOHUuL2wt7CBuYeCMczQZ40RlwbKHPQyeP9dbjale4QR6qjlUHqBkbShetK34NLnc9Cw4x4IgtLOeLqC_DKqiatTw=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134836.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_Z8MJ40IVLlYYFkafFqjGAXI9szs2zN3LBqAy25blshRhvAXFW0tJcZcsBu0RlYjCjEAF_AYrd2wNQW_Ahrvl4CCMTUMJrgmEW0TzCz5aSlthcu5lKpnBf7BSPZnm9tqw6iOZH3ETwv7nsxjPRbUr4XznZuydeLzRlIUxSZqm2BDIGGnlYD-Mj41V6kW75GbDFcfF_ZEezbvSlidJQNKAQ-CThL1qqvfmynIwM1HnH8ENdHyrSuawAbtkeE7spCiKjTcqELyKVSwjvZQU9Gt86um1TRwlvNM1wDSUBRhPdWvNJA_gz_axkprEM5nEgQd6xa37aMVcZVwhNqTbFEnCBOwvBsP7WGDNIEXcv8Fe0JgBN4yIfd2v2Glol0Se7E7XK9XIwjdsA-saAe-g1smzsPf3YYJBVzVVQCutSF1600jTM-y9Hx7TICRkDzIWpAMG2WR-PVYziH4GSfwKrUBINlWIfz571BAAbRjYZX0Gjmgv2jruBFlvarlHSJX1ruwB2_szz0NLKvVez4w64htkJ47AcZmOhCxfsLgSV0Gmr6W68yGCGHht0Pr1E5shno4KrKeW5796o-mw1gyinq5FJEVz7isnZF_L2xvwOgPAMtZrnedKkp_64ckbplGyVpGf05JBbkoZHu0P0d59wE5kEmhX52iHTN0m9cI_JkznZ-cTMgHOPpjTJ6p4MH1mF7GAYkocdEF3xltrKgPK_0Qo5WjZ5kypW41DQDHpjhAPyVRHy4XCak_HfE_5Qcxp0dXWfraJmDVwnBjeLJkgE5oHlLzr5lDu8wPtQ4GhFFzB_jncT0rB_1uhvV-R5zgYwmmftin4w_ZuWRO4DeVg=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134828.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_aGWQiUPgzszroXQ7_iGMKYxgp9eq4ki0I-qlSbwIWUH6kAfJQMAfBJdktbT5QoUX4dMntJsWPIr4hpQJIZIo_Nrvt-i3M-MjV65XkMkRyOIF9EBkwD9db62DvEGfVZu56bEQq7rWcbVxHc_gBZoYFZeq3EWB-4P5Wq6db9ACEmGtRTrWj9tsXleEvvXWl8NG7O9Dbcdg6gyGnw6f0Beyng7W1B7t7aG4eS90-BVXljmKtkbjL8L2qXYhxQJftZYLmqW_S438jatI7u0Izfycqx9uRYUUUs_aPS9foX5wcl9jA9idCA8PAOd3l8i-CyC3CDOk3715VJXIEtFsKCEwBd8c-IqFzDMEcYcHwzN_CJsf9bTDA3uYcISBIUfqek3KoTqVgWCl4aOcney74ozvAsPuisefTu9SEXFEKbU-3TqEoJB9ufl74H831_wY3n8XE60TdzwjmOvtUo8NcvO-i10Hee1_qRWEiUchzDqHElZkxdrstBw7AJPAJGkwoq6B5fOAUN0E26dGjzirFgtGgSIOEHjD31mFk41Rfp9Cz_Pen1UTfWGsga5LWuGA3GhM0I6jQe5wuJBEwHC7sQ6wcjplbawDzFGJ3z7fTtP9EXUwQIni9LbZdlFBp6okhuUbdyM1P-0F7T_ax1PUNECwItOXv0nM4bJcvHLG3NYYs0Or6CXsIILAwHaQlMcxu6-ERxUE-WWyWGdCNddDVJd1Hkd56oV5C_yCzP1GpZFLVknb5gloxfywnFsz8wsyAu6SYwYP8jgUjTwHU42XkH7fPe592waTgGsDoNaXkpb0JaOvNUyPxJZ_oeSr2915lwtIFgulIwHmC6Bw9hhg=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134816.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_ZtW8qpHnFW_nDQWt9tI56kFubJU-zSFWWoqEE3H_KBevuz160EHW09bZCC-wsB42Bptqx2LNvt1wMu2bJAzWLxYVjtiApSCF3TusCS-EWNuyibh8A1GPtG0TF0JLtYfKG2vGX1gbPijLGLNBwPJuLOpoxElv7myFPFqKjgEoC2k-1zTV8t3ZcRYtrKDmilknlwNEnhwTCfL9A264pwJK7qVxtQZJ9yW5AmY0y0juBXT4J7FI6orF3jXKk60ImKxH7W8rYNWf2Gn_2_C6h21xBlH2-WS8Zo5WcW0G66THLMIXsfa9wac2HPiYFt2rb1Kl-5tnDyeYr1H55dGkmv3YsJ-YYu8i4Cwdgu4jnl0p33kNP3zdVZeWAH-Q7LMPtbWzHhONULf6hnxOWq8VuBtBMDm3eSc5RiEhN3BxnrQ-4-hBoJfqcAwyVi-I7p4pSHcCEF3cJWL8GDb3fERas0_hWdgAXILUhFUXmNeeppwHsWaXB4_nv9WreDv4iWGZsDioTnApCEWQLX_RX0DLT8V8qid7FlVCgb_ipIxR4PEfshiErcY94Odb-70Zsq2QsXbQHlh8zzAgO_8h9IAcjlXHlbPrJWSnrmh7xKIrWgyS77c_IAh5NQv5WY4fq61z5-Jmwwx1mdK090aJ38fX6OFj37ZD2qx8P9nPYZF_YmBgabVNFRdv85H3oJN6Bi_d-qjGl-Z0VapUypO1P97a9VzF6XdVBMITRkHJuQXAo6DoWZOOKHRgzBU5-fD3mKexmRBosmeBRzaMoQEMOJ0LNjaTiI3k5o6LJ_6ny7utTQhUNutTyJTjQYnh7fCMMvXsd35EE0DM_L6X8DWXB5RQ=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_134822.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_Z7IaLckeaelYDDRKrJW7CgUXEWx02RVHgv13AmHCsfECdyqYU9xhnf2IFsOqn78vrUspcTgNzVNuqgFvB5chIVO3tKbwOWwKBkD27qR9lUOclCQ3AscLLmEGgh5akFvzXeWZfjpXd5rIRM7-cwVwJI461N1JhsKDQiaAmVsLPJ-9IzWrOPna-QNyEGqvkZBZ-P1QP4Et2xhndHd9KqQOBVF3aFFCRbEx6cGiKLJ81a5szk7kLbntqTdAKX0itT-CQDBYuOOvgSQYYcviHqgtsKr43wnakHbO_Kf4mpNIzz6A5YnVwuARUrgu4-vbtJwxVywieeG_8WG7q37IeFfpauwJs1eNuX9j07kscdzRwx4OVzXXBWHa8ChpK9NK9xsNNEUSyWot49HohzwvlZTzH81Yi6MbikXK5-CemOaYJ0SnJXDssxkFb6vIrl6HbVtnQNFkCVD8hTB9dHDlWnh0r3PQV-lVEJMDhv-2-wUbxJBb0iixntgXOUceBItnaBSBV4H5XEW4fEQxTBcuDaWCGYg01O6SHCxWwI5mnSnRNWjTpWhS10cvYGe2JXiRNKVbM4RRlJcIWOGRWKkv3pCgjenmNCyinMFznKxnyjr26AzwvQcvqQRpgDHcWov4AoMwuLGvfcaFgrGr4jToeKoPllUIf1lBB7Oq39taxAge0wnz9XXCVvnOJH9LmBcdfZzfyZuOzbN0atsNm3IqN8EW02JL18QhhYt66jzHDgGLYInFANJlY0wui04M-Sl77n2y_w-ig1fPCYqzUvCVBUUhal01Cddg6zW8rSTndS1Oq5EPKuOfKvrXK_CCkqdqWE9ZHhVwF5GwoT_gbHOg=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_123240.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_bAFYDxfDxj76tYX4VrxmVhboQlWHN9JOJ5zWCpxblDBeosD0WDJgnKeWhoP67LsyChdIVrHwMmBhQGkyctpDr-gi3x-n7e0EFNkffNoYHktX4CgkbuNHMDe5C2yS-MbXSoyCcdndq86Nip0WrWg27KXheAkyOBxHIpWcK_60sylvBlkhX6aQPorO-Ses3G1EHB3CGQJqHok-tD9gwQZPB3IDQw_nwXLe1LWhxPTyMrl3TdpzCyO903s6Ex5yCCLd68DRzoOlv2zdAEvIFllpHFJuiX8Si_2DUSLmDZm-Ck3LGc1qf9h5kxJapfX-AKLCqog5g5zbN_EySSPdMmpIHKIK-3q5E0mphlrhNXTzGP5Yx8LgsRSyvmgSlQTy9s8b2bpy_8YMqsp4TB7LGj2vTseW5Iy1qeSlK8udEk3yvoUHu-egnsVgTaAAYXtEJOxL7tLJOhJ58_YTukm5d9ATizZJ1UsNhf4h-IqAPz5qSMGjCXwTXXijBM2tTBms90bf7kuFtx0f0uMz8AnRne5ZQ0UiXDutr2rkFv1whdYJvxc0Rn_yQWxJ_koO4fEW04EPQMIoQHQjvSLd4gy7reR-hZpirltpx9tbLzsYdI7nKgpa6DLAuxYjsAn1Yf6ozqdyQgbDNo5657GhZz_Cx2auHToMVjy7hzANu_Eod4u13BA4D7qr5Qd4MfA1whLvr4DyuCjE3XoBU9Bs8j227YFdTha-tCsbOBh1h7_uSdpOrX6LIZh2MtA1jAjn4DlgAALH3wyMUcCh0Y1U4BUzyTrFOa7h0fwBXnQ7YCXoTDKii0b-xldmwQ5QuYtOTnZBW1ChQwByQAz9XVCpaoyg=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : REDACTED_Trip/IMG_20181119_123713.jpg: Failed to copy: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_ZL-Uy5RwVExjT7Iwy04R-cLLXXGX_sK1k59DhpmP9oDlmtCINsxRgf4sk5LJ2glCm26VFyEvkZJ91pM5oWlogvuYs0Ef4oev76FYADPlRznTpZq47xIMWzLjVb3uMsnJ1Xxi1BUcoZdod3Qsd_ltdT_4w4UyEC5QCArRrdc3-et_z5mSx6uzfgactWKkrrpnPWiaNLQ_kRlaDV1Ijdg8KwaTQNgAnW_LKVEqjbIZJrRfNV5N8sBgGhYwbotLsYD6izreadCGg3hFqQF21wyJYcx2nMQZmaJ-eoC6GMgDU0aYsSY3zcKXdjRQw96O-H92ircAr9D00Rbkc0uhrubHEQJ61suh53QwcMIzGeBaUrgzw62dxbSIo1mHW-h6pCElpztFQSIeQgFBsHT9j3FeQXRsslZFDMffsC_dS6d_KPS01mOwZW-DKVc-seGBbjQwQuAmYBwYdXXRANeHNO6HiSgRYDctGnoGaChljkyByjM9gpo302O6vbn7XEXNCJI0bjb1g4DMJegsvpxLCd3SuXP74g7DPSwNl53e9O6A8pPi_AoWS_7c_FdFhwSQy43VFW4uYEmX231JB3-j8iO_VYViV8Aqg5p8E-lJ528PV_W8_RsW7dhzIs-EaX2oLca-uPMCQQTzAbH_kkFoecHa_dBU-EBryORkzaU-LyuLbfrvVe-f8M-jL5OdcVpJfnGjFdXy-VukO2Wkz-3KSFM17HGv4vd8UfHtyWf8AMKVq4rapmWUggw-NOgdLHvVTseiMuZEIqk5r_50MMVR31EFYA9FGu-zcAeiCKzrIdgljqaldLr7R2CWCOaleiHRHUyuyxOCieh3w4GHVxhw=d": context deadline exceeded
2021/03/11 19:29:02 ERROR : Local file system at /REDACTED/shared-album: not deleting files as there were IO errors
2021/03/11 19:29:02 ERROR : Local file system at /REDACTED/shared-album: not deleting directories as there were IO errors
2021/03/11 19:29:02 ERROR : Attempt 1/1 failed with 15 errors and: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_ZL-Uy5RwVExjT7Iwy04R-cLLXXGX_sK1k59DhpmP9oDlmtCINsxRgf4sk5LJ2glCm26VFyEvkZJ91pM5oWlogvuYs0Ef4oev76FYADPlRznTpZq47xIMWzLjVb3uMsnJ1Xxi1BUcoZdod3Qsd_ltdT_4w4UyEC5QCArRrdc3-et_z5mSx6uzfgactWKkrrpnPWiaNLQ_kRlaDV1Ijdg8KwaTQNgAnW_LKVEqjbIZJrRfNV5N8sBgGhYwbotLsYD6izreadCGg3hFqQF21wyJYcx2nMQZmaJ-eoC6GMgDU0aYsSY3zcKXdjRQw96O-H92ircAr9D00Rbkc0uhrubHEQJ61suh53QwcMIzGeBaUrgzw62dxbSIo1mHW-h6pCElpztFQSIeQgFBsHT9j3FeQXRsslZFDMffsC_dS6d_KPS01mOwZW-DKVc-seGBbjQwQuAmYBwYdXXRANeHNO6HiSgRYDctGnoGaChljkyByjM9gpo302O6vbn7XEXNCJI0bjb1g4DMJegsvpxLCd3SuXP74g7DPSwNl53e9O6A8pPi_AoWS_7c_FdFhwSQy43VFW4uYEmX231JB3-j8iO_VYViV8Aqg5p8E-lJ528PV_W8_RsW7dhzIs-EaX2oLca-uPMCQQTzAbH_kkFoecHa_dBU-EBryORkzaU-LyuLbfrvVe-f8M-jL5OdcVpJfnGjFdXy-VukO2Wkz-3KSFM17HGv4vd8UfHtyWf8AMKVq4rapmWUggw-NOgdLHvVTseiMuZEIqk5r_50MMVR31EFYA9FGu-zcAeiCKzrIdgljqaldLr7R2CWCOaleiHRHUyuyxOCieh3w4GHVxhw=d": context deadline exceeded
2021/03/11 19:29:02 INFO  : 
Transferred:   	   24.687G / 24.687 GBytes, 100%, 7.236 MBytes/s, ETA 0s
Errors:                15 (retrying may help)
Transferred:        18506 / 28514, 65%
Elapsed time:      59m1.5s

2021/03/11 19:29:02 Failed to sync with 15 errors: last error was: failed to open source object: Get "https://lh3.googleusercontent.com/lr/AFBm1_ZL-Uy5RwVExjT7Iwy04R-cLLXXGX_sK1k59DhpmP9oDlmtCINsxRgf4sk5LJ2glCm26VFyEvkZJ91pM5oWlogvuYs0Ef4oev76FYADPlRznTpZq47xIMWzLjVb3uMsnJ1Xxi1BUcoZdod3Qsd_ltdT_4w4UyEC5QCArRrdc3-et_z5mSx6uzfgactWKkrrpnPWiaNLQ_kRlaDV1Ijdg8KwaTQNgAnW_LKVEqjbIZJrRfNV5N8sBgGhYwbotLsYD6izreadCGg3hFqQF21wyJYcx2nMQZmaJ-eoC6GMgDU0aYsSY3zcKXdjRQw96O-H92ircAr9D00Rbkc0uhrubHEQJ61suh53QwcMIzGeBaUrgzw62dxbSIo1mHW-h6pCElpztFQSIeQgFBsHT9j3FeQXRsslZFDMffsC_dS6d_KPS01mOwZW-DKVc-seGBbjQwQuAmYBwYdXXRANeHNO6HiSgRYDctGnoGaChljkyByjM9gpo302O6vbn7XEXNCJI0bjb1g4DMJegsvpxLCd3SuXP74g7DPSwNl53e9O6A8pPi_AoWS_7c_FdFhwSQy43VFW4uYEmX231JB3-j8iO_VYViV8Aqg5p8E-lJ528PV_W8_RsW7dhzIs-EaX2oLca-uPMCQQTzAbH_kkFoecHa_dBU-EBryORkzaU-LyuLbfrvVe-f8M-jL5OdcVpJfnGjFdXy-VukO2Wkz-3KSFM17HGv4vd8UfHtyWf8AMKVq4rapmWUggw-NOgdLHvVTseiMuZEIqk5r_50MMVR31EFYA9FGu-zcAeiCKzrIdgljqaldLr7R2CWCOaleiHRHUyuyxOCieh3w4GHVxhw=d": context deadline exceeded

ncw · March 12, 2021, 11:38am

Thanks for testing that.

OK I've done that patch properly now which involved patching every backend!

Would you mind giving it another go?

v1.55.0-beta.5261.004fa1c5f.fix-context-deadline on branch fix-context-deadline (uploaded in 15-30 mins)

timevortex · March 12, 2021, 7:00pm

Thanks very much for the quick fix

Re-deployed. Will report back after next run (starting in ~5 minutes).

timevortex · March 12, 2021, 9:04pm

All good still:

2021/03/12 21:02:06 INFO  : 
Transferred:   	   14.175G / 14.175 GBytes, 100%, 5.659 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:            301116 / 301116, 100%
Transferred:         1286 / 11297, 11%
Elapsed time:      58m0.0s
Transferring:
 * REDACTED…ID_20190512_132333.mp4:  0% /off, 654.671k/s, -
 * REDACTED…ID_20190515_065814.mp4:  0% /off, 650.390k/s, -
 * REDACTED…ID_20190521_175146.mp4:  0% /off, 644.214k/s, -
 * REDACTED…ID_20190522_202954.mp4:  0% /off, 635.360k/s, -
 * REDACTED…ID_20190523_084436.mp4:  0% /off, 609.859k/s, -
 * REDACTED…ID_20190523_144729.mp4:  0% /off, 633.930k/s, -
 * REDACTED…ID_20190525_092153.mp4:  0% /off, 664.352k/s, -
 * REDACTED…ID_20190526_082912.mp4:  0% /off, 611.605k/s, -
 * REDACTED…ID_20190526_103317.mp4:  0% /off, 584.582k/s, -
 * REDACTED…ID_20190526_114626.mp4:  0% /off, 0/s, -

2021/03/12 21:02:15 INFO  : REDACTED year/VID_20190526_103317.mp4: Copied (Rcat, new)
2021/03/12 21:02:20 INFO  : REDACTED year/VID_20190526_114626.mp4: Copied (Rcat, new)
2021/03/12 21:02:27 INFO  : REDACTED year/VID_20190526_082912.mp4: Copied (Rcat, new)
2021/03/12 21:02:41 INFO  : REDACTED year/VID_20190521_175146.mp4: Copied (Rcat, new)
2021/03/12 21:02:45 INFO  : REDACTED year/VID_20190529_160434.mp4: Copied (Rcat, new)
2021/03/12 21:03:06 INFO  : 
Transferred:   	   14.519G / 14.519 GBytes, 100%, 5.664 MBytes/s, ETA 0s
Errors:                 1 (retrying may help)
Checks:            301116 / 301116, 100%
Transferred:         1296 / 11307, 11%
Elapsed time:      59m0.0s
Transferring:
 * REDACTED…ID_20190512_132333.mp4:  0% /off, 638.434k/s, -
 * REDACTED…ID_20190515_065814.mp4:  0% /off, 634.998k/s, -
 * REDACTED…ID_20190522_202954.mp4:  0% /off, 610.715k/s, -
 * REDACTED…ID_20190523_084436.mp4:  0% /off, 652.677k/s, -
 * REDACTED…ID_20190523_144729.mp4:  0% /off, 621.430k/s, -
 * REDACTED…ID_20190525_092153.mp4:  0% /off, 648.054k/s, -
 * REDACTED…ID_20190526_140131.mp4:  0% /off, 646.598k/s, -
 * REDACTED…ID_20190530_072101.mp4:  0% /off, 590.840k/s, -
 * REDACTED…ID_20190530_101729.mp4:  0% /off, 603.809k/s, -
 * REDACTED…ID_20190530_171458.mp4:  0% /off, 584.856k/s, -

2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190526_140131.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190525_092153.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190530_171458.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190523_084436.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190530_101729.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190523_144729.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190530_072101.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190522_202954.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190526_140131.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190515_065814.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 NOTICE: REDACTED year/VID_20190512_132333.mp4: Removing partially written file on error: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190530_171458.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190525_092153.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190530_101729.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190530_072101.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190523_084436.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190523_144729.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190522_202954.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190515_065814.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 INFO  : can't close account: file already closed
github.com/rclone/rclone/fs/operations.init
	/home/ro0t/Downloads/rclone/fs/operations/reopen.go:28
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5652
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.doInit
	/snap/go/7013/src/runtime/proc.go:5647
runtime.main
	/snap/go/7013/src/runtime/proc.go:191
runtime.goexit
	/snap/go/7013/src/runtime/asm_amd64.s:1374
2021/03/12 21:03:06 ERROR : REDACTED year/VID_20190512_132333.mp4: Failed to copy: context deadline exceeded
2021/03/12 21:03:06 ERROR : Local file system at /REDACTED/album: not deleting files as there were IO errors
2021/03/12 21:03:06 ERROR : Local file system at /REDACTED/album: not deleting directories as there were IO errors
2021/03/12 21:03:06 ERROR : Attempt 1/1 failed with 21 errors and: context deadline exceeded
2021/03/12 21:03:06 INFO  : 
Transferred:   	   14.520G / 14.520 GBytes, 100%, 5.664 MBytes/s, ETA 0s
Errors:                21 (retrying may help)
Checks:            301116 / 301116, 100%
Transferred:         1296 / 11297, 11%
Elapsed time:      59m0.2s

2021/03/12 21:03:06 Failed to sync with 21 errors: last error was: context deadline exceeded

ncw · March 13, 2021, 9:26am

Thanks for testing

I've merged this to master now which means it will be in the latest beta in 15-30 mins and released in v1.55