Rclone transferring a folder to GDrive twice

NewDawnFades · May 29, 2021, 5:54pm

Hi there

What is the problem you are having with rclone?

I've never ever seen rclone transferring a folder to Google Drive twice!
A few weeks ago, I was transferring a 22GB folder (with 56 files inside) and right after the transfer was over, rclone started the same process again.
So, instead of 22 GB, it said 44 GBs!
Yesterday, the same thing happened (another BluRay disc).
I exited ssh/Putty and when running the same command, rclone checked that all the files had already been transferred.

What is your rclone version (output from `rclone version`)

rclone v1.55.1

os/type: linux
os/arch: amd64
go/version: go1.16.3
go/linking: static
go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone copy -P --transfers=10

logfile

2021/05/29 22:00:56 DEBUG : rclone: Version "v1.55.1" starting with parameters ["rclone" "copy" "-P" "--transfers=10" "/SEPT301955" "SE301955" "--log-file=rclone.txt" "--log-level" "DEBUG"]
2021/05/29 22:00:56 DEBUG : Creating backend with remote "/SEPT301955"
2021/05/29 22:00:56 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:56 DEBUG : Creating backend with remote "SE301955"
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:00:57 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:01:00 DEBUG : BDMV/STREAM/00000.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:00 DEBUG : BDMV/STREAM/00000.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:00 DEBUG : BDMV/STREAM/00000.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:00 DEBUG : BDMV/STREAM/00000.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:01:00 DEBUG : BDMV/STREAM/00000.m2ts: Sending chunk 33554432 length 1245184
2021/05/29 22:01:01 DEBUG : Google drive root 'SE301955': Waiting for checks to finish
2021/05/29 22:01:01 DEBUG : Google drive root 'SE301955': Waiting for transfers to finish
2021/05/29 22:01:01 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:01 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:01 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:01 INFO  : BDMV/STREAM/00000.m2ts: Copied (new)
2021/05/29 22:01:01 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00004.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:01:02 INFO  : BDMV/MovieObject.bdmv: Copied (new)
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 33554432 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 41943040 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 50331648 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:02 INFO  : BDMV/index.bdmv: Copied (new)
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 33554432 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:01:02 DEBUG : BDMV/STREAM/00004.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:03 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:03 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:03 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:01:03 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:01:03 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 75497472 length 8388608
2021/05/29 22:01:03 DEBUG: BDMV/STREAM/00002.m2ts: Sending chunk 33554432 length 8388608
2021/05/29 22:01:04 INFO  : BDMV/STREAM/00008.m2ts: Copied (new)
2021/05/29 22:01:04 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:01:04 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 117440512 length 8388608
2021/05/29 22:01:04 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:01:04 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 125829120 length 8388608
2021/05/29 22:01:04 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 41943040 length 8388608
2021/05/29 22:01:04 DEBUG :BDMV/STREAM/00001.m2ts: Sending chunk 176160768 length 8388608
2021/05/29 22:01:05 INFO  : CERTIFICATE/id.bdmv: Copied (new)
2021/05/29 22:01:05 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 33554432 length 8388608
2021/05/29 22:01:05 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 184549376 length 8388608
2021/05/29 22:01:05 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 41943040 length 8388608
2021/05/29 22:01:05 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 192937984 length 8388608
2021/05/29 22:01:05 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:05 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:05 DEBUG : pacer: Rate limited, increasing sleep to 1.432632513s
2021/05/29 22:01:05 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:05 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:05 DEBUG : pacer: Rate limited, increasing sleep to 1.695897877s
2021/05/29 22:01:05 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:06 INFO  : CERTIFICATE/BACKUP/id.bdmv: Copied (new)
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00004.m2ts: Sending chunk 41943040 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 201326592 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 50331648 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 41943040 length 8388608
2021/05/29 22:01:07 DEBUG : BDMV/STREAM/00003.m2ts: Sending chunk 75497472 length 7471104
2021/05/29 22:01:07 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:07 DEBUG : pacer: Rate limited, increasing sleep to 1.995066854s
2021/05/29 22:01:07 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:07 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:07 DEBUG : pacer: Rate limited, increasing sleep to 1.409267566s
2021/05/29 22:01:07 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:08 INFO  : BDMV/STREAM/00003.m2ts: Copied (new)
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 209715200 length 8388608
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00004.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00005.m2ts: Sending chunk 83886080 length 851968
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 58720256 length 8388608
2021/05/29 22:01:09 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:09 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:09 DEBUG : pacer: Rate limited, increasing sleep to 1.461022293s
2021/05/29 22:01:09 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:09 DEBUG : pacer: low level retry 6/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:09 DEBUG : pacer: Rate limited, increasing sleep to 1.045723673s
2021/05/29 22:01:10 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:10 INFO  : BDMV/STREAM/00005.m2ts: Copied (new)
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 75497472 length 8388608
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 75497472 length 8388608
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 218103808 length 8388608
2021/05/29 22:01:10 DEBUG : BDMV/STREAM/00004.m2ts: Sending chunk 67108864 length 8388608
2021/05/29 22:01:10 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:10 DEBUG : pacer: Rate limited, increasing sleep to 1.601173493s
2021/05/29 22:01:10 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:11 DEBUG : pacer: low level retry 8/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:11 DEBUG : pacer: Rate limited, increasing sleep to 1.994596492s
2021/05/29 22:01:11 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:12 DEBUG : BDMV/STREAM/00006.m2ts: Sending chunk 75497472 length 8388608
2021/05/29 22:01:12 DEBUG : BDMV/STREAM/00002.m2ts: Sending chunk 83886080 length 7929856
2021/05/29 22:01:12 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 285212672 length 8388608
2021/05/29 22:01:13 INFO  : BDMV/STREAM/00002.m2ts: Copied (new)
2021/05/29 22:01:13 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 142606336 length 8388608
2021/05/29 22:01:13 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 293601280 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 301989888 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 150994944 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 150994944 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 310378496 length 8388608
2021/05/29 22:01:14 INFO  : BDMV/STREAM/00004.m2ts: Copied (new)
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 159383552 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00007.m2ts: Sending chunk 159383552 length 4390912
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 318767104 length 8388608
2021/05/29 22:01:14 INFO  : BDMV/PLAYLIST/00003.mpls: Copied (new)
2021/05/29 22:01:14 INFO  : BDMV/PLAYLIST/00000.mpls: Copied (new)
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 167772160 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 327155712 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 176160768 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 335544320 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 184549376 length 8388608
2021/05/29 22:01:14 INFO  : BDMV/PLAYLIST/00002.mpls: Copied (new)
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 343932928 length 8388608
2021/05/29 22:01:14 INFO  : BDMV/STREAM/00006.m2ts: Copied (new)
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 192937984 length 8388608
2021/05/29 22:01:14 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 352321536 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 201326592 length 8388608
2021/05/29 22:01:15 INFO  : BDMV/PLAYLIST/00004.mpls: Copied (new)
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 360710144 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 209715200 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 369098752 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 218103808 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 377487360 length 8388608
2021/05/29 22:01:15 INFO  : BDMV/PLAYLIST/00001.mpls: Copied (new)
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 226492416 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 385875968 length 8388608
2021/05/29 22:01:15 INFO  : BDMV/STREAM/00007.m2ts: Copied (new)
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 234881024 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 394264576 length 8388608
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 243269632 length 8388608
2021/05/29 22:01:15 INFO  : BDMV/PLAYLIST/00005.mpls: Copied (new)
2021/05/29 22:01:15 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 402653184 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 411041792 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 251658240 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 419430400 length 8388608
2021/05/29 22:01:16 INFO  : BDMV/PLAYLIST/00008.mpls: Copied (new)
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 260046848 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 427819008 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 268435456 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 436207616 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 276824064 length 8388608
2021/05/29 22:01:16 INFO  : BDMV/PLAYLIST/00007.mpls: Copied (new)
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 285212672 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 444596224 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 452984832 length 8388608
2021/05/29 22:01:16 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 293601280 length 8388608
2021/05/29 22:01:16 INFO  : BDMV/PLAYLIST/00006.mpls: Copied (new)
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 461373440 length 8388608
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00009.m2ts: Sending chunk 301989888 length 1376256
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 469762048 length 8388608
2021/05/29 22:01:17 INFO  : BDMV/BACKUP/MovieObject.bdmv: Copied (new)
2021/05/29 22:01:17 INFO  : BDMV/PLAYLIST/00009.mpls: Copied (new)
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 478150656 length 8388608
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 486539264 length 8388608
2021/05/29 22:01:17 INFO  : BDMV/BACKUP/index.bdmv: Copied (new)
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 494927872 length 8388608
2021/05/29 22:01:17 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 503316480 length 8388608
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 511705088 length 8388608
2021/05/29 22:01:18 INFO  : BDMV/STREAM/00009.m2ts: Copied (new)
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 520093696 length 8388608
2021/05/29 22:01:18 INFO  : BDMV/BACKUP/PLAYLIST/00000.mpls: Copied (new)
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 528482304 length 8388608
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 536870912 length 8388608
2021/05/29 22:01:18 INFO  : BDMV/BACKUP/PLAYLIST/00005.mpls: Copied (new)
2021/05/29 22:01:18 INFO  : BDMV/BACKUP/PLAYLIST/00006.mpls: Copied (new)
2021/05/29 22:01:18 INFO  : BDMV/BACKUP/PLAYLIST/00004.mpls: Copied (new)
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 545259520 length 8388608
2021/05/29 22:01:18 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 553648128 length 8388608
2021/05/29 22:01:19 INFO  : BDMV/BACKUP/PLAYLIST/00007.mpls: Copied (new)
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 562036736 length 8388608
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 570425344 length 8388608
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 578813952 length 8388608
2021/05/29 22:01:19 INFO  : BDMV/BACKUP/PLAYLIST/00003.mpls: Copied (new)
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 587202560 length 8388608
2021/05/29 22:01:19 INFO  : BDMV/BACKUP/PLAYLIST/00002.mpls: Copied (new)
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 595591168 length 8388608
2021/05/29 22:01:19 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 603979776 length 8388608
2021/05/29 22:01:20 INFO  : BDMV/BACKUP/PLAYLIST/00001.mpls: Copied (new)
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 612368384 length 8388608
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 620756992 length 8388608
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 629145600 length 8388608
2021/05/29 22:01:20 INFO  : BDMV/BACKUP/PLAYLIST/00009.mpls: Copied (new)
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 637534208 length 8388608
2021/05/29 22:01:20 INFO  : BDMV/BACKUP/PLAYLIST/00008.mpls: Copied (new)
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 645922816 length 8388608
2021/05/29 22:01:20 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 654311424 length 8388608
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 662700032 length 8388608
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 671088640 length 8388608
2021/05/29 22:01:21 INFO  : BDMV/BACKUP/CLIPINF/00000.clpi: Copied (new)
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 679477248 length 8388608
2021/05/29 22:01:21 INFO  : BDMV/BACKUP/CLIPINF/00004.clpi: Copied (new)
2021/05/29 22:01:21 INFO  : BDMV/BACKUP/CLIPINF/00002.clpi: Copied (new)
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 687865856 length 8388608
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 696254464 length 8388608
2021/05/29 22:01:21 INFO  : BDMV/BACKUP/CLIPINF/00008.clpi: Copied (new)
2021/05/29 22:01:21 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 704643072 length 8388608
2021/05/29 22:01:22 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 713031680 length 8388608
2021/05/29 22:01:22 INFO  : BDMV/BACKUP/CLIPINF/00003.clpi: Copied (new)
2021/05/29 22:01:22 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 721420288 length 8388608
2021/05/29 22:01:22 INFO  : BDMV/BACKUP/CLIPINF/00001.clpi: Copied (new)
2021/05/29 22:01:22 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 729808896 length 8388608
2021/05/29 22:01:22 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 738197504 length 8388608
2021/05/29 22:01:22 INFO  : BDMV/BACKUP/CLIPINF/00006.clpi: Copied (new)
2021/05/29 22:01:22 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 746586112 length 8388608
2021/05/29 22:01:22 INFO  : BDMV/BACKUP/CLIPINF/00005.clpi: Copied (new)
2021/05/29 22:01:23 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 754974720 length 8388608
2021/05/29 22:01:23 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 763363328 length 8388608
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 119.304µs
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 89.796635ms
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 83.397221ms
2021/05/29 22:01:23 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 771751936 length 8388608
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 100.315988ms
2021/05/29 22:01:23 INFO  : BDMV/BACKUP/CLIPINF/00007.clpi: Copied (new)
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 146.917833ms
2021/05/29 22:01:23 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 780140544 length 8388608
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 94.419989ms
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 100.130908ms
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 176.235695ms
2021/05/29 22:01:23 INFO  : BDMV/BACKUP/CLIPINF/00009.clpi: Copied (new)
2021/05/29 22:01:23 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 788529152 length 8388608
2021/05/29 22:01:23 DEBUG : pacer: Reducing sleep to 132.406529ms
2021/05/29 22:01:24 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:24 DEBUG : pacer: Rate limited, increasing sleep to 1.507260139s
2021/05/29 22:01:24 DEBUG : pacer: Reducing sleep to 49.519239ms
2021/05/29 22:01:24 INFO  : BDMV/CLIPINF/00000.clpi: Copied (new)
2021/05/29 22:01:24 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 796917760 length 8388608
2021/05/29 22:01:24 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:24 DEBUG : pacer: Rate limited, increasing sleep to 1.256394134s
2021/05/29 22:01:24 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:24 DEBUG : pacer: Rate limited, increasing sleep to 2.006482563s
2021/05/29 22:01:24 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:24 DEBUG : pacer: Reducing sleep to 26.082069ms
2021/05/29 22:01:24 INFO  : BDMV/CLIPINF/00002.clpi: Copied (new)
2021/05/29 22:01:24 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:24 DEBUG : pacer: Rate limited, increasing sleep to 1.135209787s
2021/05/29 22:01:24 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:24 INFO  : BDMV/CLIPINF/00005.clpi: Copied (new)
2021/05/29 22:01:24 INFO  : BDMV/CLIPINF/00004.clpi: Copied (new)
2021/05/29 22:01:25 INFO  : BDMV/CLIPINF/00001.clpi: Copied (new)
2021/05/29 22:01:25 INFO  : BDMV/CLIPINF/00006.clpi: Copied (new)
2021/05/29 22:01:25 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 805306368 length 8388608
2021/05/29 22:01:25 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:25 DEBUG : pacer: Rate limited, increasing sleep to 1.633101623s
2021/05/29 22:01:25 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:25 DEBUG : pacer: Rate limited, increasing sleep to 2.15300631s
2021/05/29 22:01:25 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:25 INFO  : BDMV/CLIPINF/00003.clpi: Copied (new)
2021/05/29 22:01:25 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:25 DEBUG : pacer: Rate limited, increasing sleep to 1.275143303s
2021/05/29 22:01:26 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:26 INFO  : BDMV/CLIPINF/00007.clpi: Copied (new)
2021/05/29 22:01:27 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 813694976 length 8388608
2021/05/29 22:01:27 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:27 DEBUG : pacer: Rate limited, increasing sleep to 1.569788127s
2021/05/29 22:01:27 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:27 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:27 DEBUG : pacer: Rate limited, increasing sleep to 1.651449707s
2021/05/29 22:01:27 DEBUG : pacer: low level retry 6/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:01:27 DEBUG : pacer: Rate limited, increasing sleep to 2.40821058s
2021/05/29 22:01:28 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 822083584 length 8388608
2021/05/29 22:01:29 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:01:31 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 830472192 length 8388608
2021/05/29 22:01:31 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 838860800 length 8388608
2021/05/29 22:01:31 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 847249408 length 8388608
2021/05/29 22:01:31 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 855638016 length 8388608
2021/05/29 22:01:31 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 864026624 length 8388608
2021/05/29 22:01:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 872415232 length 8388608
2021/05/29 22:01:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 880803840 length 8388608
2021/05/29 22:01:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 889192448 length 8388608
2021/05/29 22:01:32 INFO  : BDMV/CLIPINF/00009.clpi: Copied (new)
2021/05/29 22:01:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 897581056 length 8388608
2021/05/29 22:01:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 905969664 length 8388608
2021/05/29 22:01:32 INFO  : BDMV/CLIPINF/00008.clpi: Copied (new)
2021/05/29 22:01:33 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3909091328 length 8388608
2021/05/29 22:02:32 DEBUG : /SEPT301955: closing 13 unused connections
2021/05/29 22:02:32 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3917479936 length 8388608
2021/05/29 22:02:33 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3925868544 length 8388608
2021/05/29 22:02:33 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3934257152 length 8388608
2021/05/29 22:02:33 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3942645760 length 8388608
2021/05/29 22:02:33 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 3951034368 length 8388608
2021/05/29 22:02:33 DEBUG : BDMV/STREAM/00001.m2ts: Received error: read tcp [2001:1af8:4700:b0e0:e600::]:41112->[2001:1af8:4700:b0e0:e600::]:21: i/o timeout - low level retry 1/10
2021/05/29 22:09:02 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:09:02 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:09:03 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:09:03 DEBUG : BDMV/STREAM/00001.m2ts: Received error: read tcp [2001:1af8:4700:b0e0:e600::]:59736->[2001:1af8:4700:b0e0:e600::]:21: i/o timeout - low level retry 2/10
2021/05/29 22:16:47 DEBUG : /SEPT301955: Connecting to FTP server
2021/05/29 22:16:47 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:16:47 DEBUG : pacer: Rate limited, increasing sleep to 1.443756038s
2021/05/29 22:16:47 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:16:47 DEBUG : pacer: Rate limited, increasing sleep to 2.821823314s
2021/05/29 22:16:49 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/05/29 22:16:49 DEBUG : pacer: Rate limited, increasing sleep to 4.63675409s
2021/05/29 22:16:52 DEBUG : pacer: Reducing sleep to 0s
2021/05/29 22:16:57 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 0 length 8388608
2021/05/29 22:16:57 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 8388608 length 8388608
2021/05/29 22:16:57 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 16777216 length 8388608
2021/05/29 22:16:57 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 25165824 length 8388608
2021/05/29 22:16:57 DEBUG : BDMV/STREAM/00001.m2ts: Sending chunk 33554432 length 8388608
2021/05/29 22:16:57 DEBUG : ```

Animosity022 · May 29, 2021, 5:58pm

Without the most critical piece, which is the log file, there's no way to assist.

You have to include a log file with -vv as that'll will show exactly why something is being copied.

NewDawnFades · May 29, 2021, 8:28pm

Thanks for the heads up.

DEBUG logfile added to the OP.
BTW rclone started transferring the same folder even after 2 runs.

asdffdsa · May 29, 2021, 8:35pm

hi,

each time rclone copy is run, it has to check the source to the dest, looking for changed files.

that debug log looks strange to me as the source and dest appear to be local.
as a remote needs to have a colon : in it.

in the log, which file(s) are copied twice?

can you post the config file?

NewDawnFades · May 29, 2021, 8:37pm

I edited the original logfile and removed both any reference to the source (seedbox) and dest (GDrive). I guess I accidentally removed : too.

Looks like it's only happening when transferring BluRays (large files) even tho I had no probs whatsoever transferring them prior to updating rclone.

asdffdsa · May 29, 2021, 8:41pm

what version of rclone were you using in the past?

so in that log, what files are being copied twice?

NewDawnFades · May 29, 2021, 8:48pm

v1.55.0 (started using rclone back in 2019).

The files are not copied twice (to the dest folder): they are transferred twice or even more, as if in a loop.

rclone starts transferring them again after this error
Received error: read tcp [2001:1af8:4700:b0e0:e600::]:59736->[2001:1af8:4700:b0e0:e600::]:21: i/o timeout - low level retry 2

asdffdsa · May 29, 2021, 8:54pm

as i understand it, if for some reason rclone fails to copy a file, it will try again.
by default three times.

in the past, using the older version of the rclone, you used the exact same two remotes?
the only thing that changed was the rclone version?

all the seedboxes i ever used, support sftp, which is more much reliable.

NewDawnFades · May 29, 2021, 9:17pm

The weird thing is that rclone does transfer the files.

Yes, I've been using the same remotes. Earlier this year, I transferred several BluRays the same day with no problems.

My seedbox supports sftp too but so I might as well try it myself instead of using FTP.
Thanks.

asdffdsa · May 29, 2021, 9:20pm

i get much better results with sftp verus ftp.

well, it could be a new issue, something to do with the seedbox, something to do with the internet connection, etc.

system · July 29, 2021, 5:20pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.