My Rclone sync is resulting in multiples of large files. How do I prevent this?

Help and Support
1

Hi all, I am running the following Rclone script which is a sync of my seedbox, however it is resulting in certain large files being transferred multiple times over and finally resulting in a trigger of Google’s 750 gig 24 hour ban. I have also included a chunk of the log below pertaining to one of the large files that was eventually duped 11 times!

Is there a problem with my code? Or should I be looking elsewhere to solve this issue.

#!/bin/bash if pidof -o %PPID -x “rclone-cron.sh”; then exit 1 fi /media/dma/craftyclown/bin/rclone sync ~/private/rtorrent/data/ "gdrive:/The Skull/Feral" -v --min-age 1m --log-file=/media/dma/craftyclown/rclone-upload.log

* * * * * /media/dma/craftyclown/rclone-cron.sh >/dev/null 2>&1

[CODE]17 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc
2018/06/11 17:02:02 INFO :
Transferred: 23.804 GBytes (28.991 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 14m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 74% /31.928G, 16.857M/s, 8m13s

2018/06/11 17:02:02 INFO :
Transferred: 29.667 GBytes (25.298 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 20m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 92% /31.928G, 18.903M/s, 2m2s

2018/06/11 17:02:02 INFO :
Transferred: 13.961 GBytes (26.413 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 9m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 43% /31.928G, 28.309M/s, 10m49s

2018/06/11 17:02:02 INFO :
Transferred: 8.203 GBytes (23.278 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 6m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 25% /31.928G, 22.671M/s, 17m51s

2018/06/11 17:02:02 INFO :
Transferred: 17.469 GBytes (27.062 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 11m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 54% /31.928G, 19.379M/s, 12m44s

2018/06/11 17:02:02 INFO :
Transferred: 29.672 GBytes (28.108 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 18m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 92% /31.928G, 19.278M/s, 1m59s

2018/06/11 17:02:03 INFO :
Transferred: 15.036 GBytes (25.629 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 10m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 47% /31.928G, 18.576M/s, 15m31s

2018/06/11 17:02:03 INFO :
Transferred: 18.875 GBytes (26.796 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 12m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 59% /31.928G, 21.564M/s, 10m19s

2018/06/11 17:02:03 INFO :
Transferred: 29.685 GBytes (29.749 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 17m1.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 92% /31.928G, 18.951M/s, 2m1s

runtime stack:
runtime.throw(0x1024803, 0x9)
/opt/go/go1.10/src/runtime/panic.go:619 +0x81
runtime.newosproc(0xc420556000, 0xc42054a000)
/opt/go/go1.10/src/runtime/os_linux.go:164 +0x1af
runtime.newm1(0xc420556000)
/opt/go/go1.10/src/runtime/proc.go:1874 +0x113
runtime.newm(0x107efd0, 0xc420053900)
/opt/go/go1.10/src/runtime/proc.go:1853 +0x9b
runtime.startm(0x0, 0xc42054df01)
/opt/go/go1.10/src/runtime/proc.go:1997 +0x138
runtime.wakep()
/opt/go/go1.10/src/runtime/proc.go:2078 +0x48
runtime.resetspinning()
/opt/go/go1.10/src/runtime/proc.go:2449 +0x71
runtime.schedule()
/opt/go/go1.10/src/runtime/proc.go:2543 +0x125
runtime.mstart1(0x0)
/opt/go/go1.10/src/runtime/proc.go:1232 +0x9e
runtime.mstart()
/opt/go/go1.10/src/runtime/proc.go:1188 +0x76

goroutine 1 [runnable]:
github.com/ncw/rclone/fs/sync.newSyncCopyMove(0x113ede0, 0xc42029c360, 0x113f360, 0xc420030e00, 0x100, 0x7ffc70eecf03, 0xc4204e9958, 0x87440d)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync/sync.go:76 +0xf9
github.com/ncw/rclone/fs/sync.runSyncCopyMove(0x113ede0, 0xc42029c360, 0x113f360, 0xc420030e00, 0x100, 0x0, 0x113abe0)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync/sync.go:777 +0x10b
github.com/ncw/rclone/fs/sync.moveDir(0x113ede0, 0xc42029c360, 0x113f360, 0xc420030e00, 0xc420030e00, 0xe86a80, 0x60)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync/sync.go:796 +0x59
github.com/ncw/rclone/fs/sync.MoveDir(0x113ede0, 0xc42029c360, 0x113f360, 0xc420030e00, 0xc4204e9a00, 0x7985f6, 0x8)
/home/ncw/go/src/github.com/ncw/rclone/fs/sync/sync.go:835 +0x309
github.com/ncw/rclone/cmd/moveto.glob..func1.1(0xc4212c6ae0, 0xc4204e9b10)
/home/ncw/go/src/github.com/ncw/rclone/cmd/moveto/moveto.go:53 +0x63
github.com/ncw/rclone/cmd.Run(0xc420330101, 0x16eea80, 0xc4204e9c88)
/home/ncw/go/src/github.com/ncw/rclone/cmd/cmd.go:293 +0xa7
github.com/ncw/rclone/cmd/moveto.glob..func1(0x16eea80, 0xc42033c190, 0x2, 0x5)
/home/ncw/go/src/github.com/ncw/rclone/cmd/moveto/moveto.go:51 +0x154
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).execute(0x16eea80, 0xc42033c140, 0x5, 0x5, 0x16eea80, 0xc42033c140)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:760 +0x2c1
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x16e8920, 0xd74dce, 0xc42022bf78, 0x4042ac)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:846 +0x30a
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).Execute(0x16e8920, 0xc42022bf58, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:794 +0x2b
main.main()
/home/ncw/go/src/github.com/ncw/rclone/rclone.go:16 +0x31

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
/opt/go/go1.10/src/runtime/sigqueue.go:139 +0xa6
os/signal.loop()
/opt/go/go1.10/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
/opt/go/go1.10/src/os/signal/signal_unix.go:28 +0x41

goroutine 85 [select]:
github.com/ncw/rclone/cmd.StartStats.func1(0xc4212c6ae0)
/home/ncw/go/src/github.com/ncw/rclone/cmd/cmd.go:377 +0xf6
created by github.com/ncw/rclone/cmd.StartStats
/home/ncw/go/src/github.com/ncw/rclone/cmd/cmd.go:374 +0x76

goroutine 82 [IO wait]:
internal/poll.runtime_pollWait(0x7f754ff06f00, 0x72, 0xc420422668)
/opt/go/go1.10/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc4203e6b98, 0x72, 0xffffffffffffff00, 0x112fa40, 0x16865f0)
/opt/go/go1.10/src/internal/poll/fd_poll_runtime.go:85 +0x9b
internal/poll.(*pollDesc).waitRead(0xc4203e6b98, 0xc42042c400, 0x400, 0x400)
/opt/go/go1.10/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc4203e6b80, 0xc42042c400, 0x400, 0x400, 0x0, 0x0, 0x0)
/opt/go/go1.10/src/internal/poll/fd_unix.go:157 +0x17d
net.(*netFD).Read(0xc4203e6b80, 0xc42042c400, 0x400, 0x400, 0xc4204227e8, 0x41084d, 0xc42137e4b0)
/opt/go/go1.10/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc420336008, 0xc42042c400, 0x400, 0x400, 0x0, 0x0, 0x0)
/opt/go/go1.10/src/net/net.go:176 +0x6a
net.(Conn).Read-fm(0xc42042c400, 0x400, 0x400, 0x7f754fcc4c64, 0x4270ce, 0x107f110)
/home/ncw/go/src/github.com/ncw/rclone/fs/fshttp/http.go:84 +0x4d
github.com/ncw/rclone/fs/fshttp.(*timeoutConn).readOrWrite(0xc420332040, 0xc420422858, 0xc42042c400, 0x400, 0x400, 0xc420406280, 0xc42032e0f0, 0x1)
/home/ncw/go/src/github.com/ncw/rclone/fs/fshttp/http.go:72 +0x48
github.com/ncw/rclone/fs/fshttp.(*timeoutConn).Read(0xc420332040, 0xc42042c400, 0x400, 0x400, 0xf8e240, 0x0, 0x7f754fe86000)
/home/ncw/go/src/github.com/ncw/rclone/fs/fshttp/http.go:84 +0x82
crypto/tls.(*block).readFromUntil(0xc42032e0f0, 0x7f754fe86000, 0xc420332040, 0x5, 0xc420332040, 0xc4200435d0)
/opt/go/go1.10/src/crypto/tls/conn.go:493 +0x96
crypto/tls.(*Conn).readRecord(0xc420406000, 0x107f117, 0xc420406120, 0x454ba0)
/opt/go/go1.10/src/crypto/tls/conn.go:595 +0xe0
crypto/tls.(*Conn).Read(0xc420406000, 0xc4213ae000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/opt/go/go1.10/src/crypto/tls/conn.go:1156 +0x100
net/http.(*persistConn).Read(0xc42023bb00, 0xc4213ae000, 0x1000, 0x1000, 0xc420422b98, 0x403805, 0xc4212c6900)
/opt/go/go1.10/src/net/http/transport.go:1453 +0x136
bufio.(*Reader).fill(0xc421263080)
/opt/go/go1.10/src/bufio/bufio.go:100 +0x11e
bufio.(*Reader).Peek(0xc421263080, 0x1, 0x0, 0x0, 0x1, 0xc4212c69c0, 0x0)
/opt/go/go1.10/src/bufio/bufio.go:132 +0x3a
net/http.(*persistConn).readLoop(0xc42023bb00)
/opt/go/go1.10/src/net/http/transport.go:1601 +0x185
created by net/http.(*Transport).dialConn
/opt/go/go1.10/src/net/http/transport.go:1237 +0x95a

goroutine 83 [select]:
net/http.(*persistConn).writeLoop(0xc42023bb00)
/opt/go/go1.10/src/net/http/transport.go:1822 +0x14b
created by net/http.(*Transport).dialConn
/opt/go/go1.10/src/net/http/transport.go:1238 +0x97f
2018/06/11 17:03:02 INFO :
Transferred: 24.947 GBytes (25.024 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 17m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 78% /31.928G, 21.637M/s, 5m30s

runtime: failed to create new OS thread (have 4 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
runtime: failed to create new OS thread (have 5 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc
fatal error: newosproc

runtime stack:
runtime.throw(0x1024803, 0x9)
/opt/go/go1.10/src/runtime/panic.go:619 +0x81
runtime.newosproc(0xc4200c6000, 0xc4200d8000)
/opt/go/go1.10/src/runtime/os_linux.go:164 +0x1af
runtime.newm1(0xc4200c6000)
/opt/go/go1.10/src/runtime/proc.go:1874 +0x113
runtime.newm(0x107efd0, 0xc42003ea00)
/opt/go/go1.10/src/runtime/proc.go:1853 +0x9b
runtime.startm(0x0, 0xc4200a3f01)
/opt/go/go1.10/src/runtime/proc.go:1997 +0x138
runtime.wakep()
/opt/go/go1.10/src/runtime/proc.go:2078 +0x48
runtime.resetspinning()
/opt/go/go1.10/src/runtime/proc.go:2449 +0x71
runtime.schedule()
/opt/go/go1.10/src/runtime/proc.go:2543 +0x125
runtime.mstart1(0x0)
/opt/go/go1.10/src/runtime/proc.go:1232 +0x9e
runtime.mstart()
/opt/go/go1.10/src/runtime/proc.go:1188 +0x76

goroutine 1 [runnable, locked to thread]:
runtime.gcenable()
/opt/go/go1.10/src/runtime/mgc.go:214 +0x85
runtime.main()
/opt/go/go1.10/src/runtime/proc.go:161 +0x11e
runtime.goexit()
/opt/go/go1.10/src/runtime/asm_amd64.s:2361 +0x1

runtime stack:
runtime.throw(0x1024803, 0x9)
/opt/go/go1.10/src/runtime/panic.go:619 +0x81
runtime.newosproc(0xc420098800, 0xc4200a6000)
/opt/go/go1.10/src/runtime/os_linux.go:164 +0x1af
runtime.newm1(0xc420098800)
/opt/go/go1.10/src/runtime/proc.go:1874 +0x113
runtime.newm(0x0, 0xc42003a000)
/opt/go/go1.10/src/runtime/proc.go:1853 +0x9b
runtime.startm(0xc42003a000, 0x16fda00)
/opt/go/go1.10/src/runtime/proc.go:1997 +0x138
runtime.handoffp(0xc42003a000)
/opt/go/go1.10/src/runtime/proc.go:2024 +0x55
runtime.stoplockedm()
/opt/go/go1.10/src/runtime/proc.go:2092 +0x101
runtime.schedule()
/opt/go/go1.10/src/runtime/proc.go:2488 +0x2da
runtime.goschedImpl(0xc420000180)
/opt/go/go1.10/src/runtime/proc.go:2614 +0xf7
runtime.gopreempt_m(0xc420000180)
/opt/go/go1.10/src/runtime/proc.go:2642 +0x36
runtime.newstack()
/opt/go/go1.10/src/runtime/stack.go:1046 +0x2aa
runtime.morestack()
/opt/go/go1.10/src/runtime/asm_amd64.s:480 +0x89
2018/06/11 17:03:02 INFO :
Transferred: 24.961 GBytes (26.601 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 16m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 78% /31.928G, 21.869M/s, 5m26s

2018/06/11 17:03:02 INFO :
Transferred: 24.964 GBytes (28.379 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 15m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 78% /31.928G, 21.938M/s, 5m25s

2018/06/11 17:03:02 INFO :
Transferred: 30.953 GBytes (25.139 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 21m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 96% /31.928G, 22.186M/s, 45s

2018/06/11 17:03:02 INFO :
Transferred: 15.734 GBytes (26.797 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 10m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 49% /31.928G, 29.836M/s, 9m15s

2018/06/11 17:03:02 INFO :
Transferred: 9.945 GBytes (24.198 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 7m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 31% /31.928G, 29.765M/s, 12m36s

2018/06/11 17:03:02 INFO :
Transferred: 18.836 GBytes (26.752 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 12m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 58% /31.928G, 24.439M/s, 9m8s

2018/06/11 17:03:02 INFO :
Transferred: 30.965 GBytes (27.791 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 19m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 96% /31.928G, 22.339M/s, 44s

2018/06/11 17:03:03 INFO :
Transferred: 16.094 GBytes (24.941 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 11m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 50% /31.928G, 10.262M/s, 26m20s

2018/06/11 17:03:03 INFO :
Transferred: 20.500 GBytes (26.868 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 13m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 64% /31.928G, 27.779M/s, 7m1s

2018/06/11 17:03:03 INFO :
Transferred: 30.984 GBytes (29.329 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 18m1.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 97% /31.928G, 22.533M/s, 42s

2018/06/11 17:03:53 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:03:54 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:03:54 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:04:01 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Deleted
2018/06/11 17:04:01 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Couldn’t delete: remove /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:04:01 INFO :
Transferred: 31.928 GBytes (28.701 MBytes/s)
Errors: 0
Checks: 1
Transferred: 1
Elapsed time: 18m59.1s

2018/06/11 17:04:01 ERROR : Attempt 1/3 failed with 1 errors and: remove /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:04:01 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Couldn’t delete: remove /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:04:01 ERROR : Attempt 1/3 failed with 1 errors and: remove /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:04:01 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:04:01 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:04:01 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:04:01 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:04:02 INFO :
Transferred: 26.188 GBytes (24.810 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 18m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 82% /31.928G, 22.395M/s, 4m22s

2018/06/11 17:04:02 INFO :
Transferred: 26.195 GBytes (26.276 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 17m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 82% /31.928G, 22.256M/s, 4m23s

2018/06/11 17:04:02 INFO :
Transferred: 26.195 GBytes (27.919 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 16m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 82% /31.928G, 22.332M/s, 4m22s

2018/06/11 17:04:02 INFO :
Transferred: 31.928 GBytes (24.753 MBytes/s)
Errors: 0
Checks: 1
Transferred: 0
Elapsed time: 22m0.8s

2018/06/11 17:04:02 INFO :
Transferred: 17.211 GBytes (26.652 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 11m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 53% /31.928G, 24.604M/s, 10m12s

2018/06/11 17:04:02 INFO :
Transferred: 11.672 GBytes (24.856 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 8m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 36% /31.928G, 31.679M/s, 10m54s

2018/06/11 17:04:02 INFO :
Transferred: 20.234 GBytes (26.530 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 13m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 63% /31.928G, 23.399M/s, 8m31s

2018/06/11 17:04:02 INFO :
Transferred: 31.928 GBytes (27.224 MBytes/s)
Errors: 0
Checks: 1
Transferred: 0
Elapsed time: 20m0.9s

2018/06/11 17:04:03 INFO :
Transferred: 17.227 GBytes (24.474 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 12m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 53% /31.928G, 23.002M/s, 10m54s

2018/06/11 17:04:03 INFO :
Transferred: 22.020 GBytes (26.801 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 14m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 68% /31.928G, 28.255M/s, 5m59s

2018/06/11 17:04:31 INFO : Google drive root ‘The Skull/Complete’: Waiting for checks to finish
2018/06/11 17:04:31 INFO : Google drive root ‘The Skull/Complete’: Waiting for transfers to finish
2018/06/11 17:04:31 ERROR : Attempt 2/3 succeeded
2018/06/11 17:04:31 INFO :
Transferred: 31.928 GBytes (24.219 MBytes/s)
Errors: 0
Checks: 1
Transferred: 0
Elapsed time: 22m29.9s

2018/06/11 17:04:49 INFO : Google drive root ‘The Skull/Complete’: Waiting for checks to finish
2018/06/11 17:04:49 INFO : Google drive root ‘The Skull/Complete’: Waiting for transfers to finish
2018/06/11 17:04:49 ERROR : Attempt 2/3 succeeded
2018/06/11 17:04:49 INFO :
Transferred: 31.928 GBytes (26.203 MBytes/s)
Errors: 0
Checks: 1
Transferred: 0
Elapsed time: 20m47.7s

2018/06/11 17:04:51 INFO : Google drive root ‘The Skull/Complete’: Modify window is 1ms
2018/06/11 17:04:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:04:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:05:02 INFO :
Transferred: 27.656 GBytes (24.823 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 19m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 86% /31.928G, 26.771M/s, 2m43s

2018/06/11 17:05:02 INFO :
Transferred: 27.785 GBytes (26.323 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 18m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 87% /31.928G, 29.796M/s, 2m22s

2018/06/11 17:05:02 INFO :
Transferred: 27.789 GBytes (27.877 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 17m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 87% /31.928G, 29.574M/s, 2m23s

2018/06/11 17:05:02 INFO :
Transferred: 19 GBytes (26.975 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 12m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 59% /31.928G, 31.363M/s, 7m2s

2018/06/11 17:05:02 INFO :
Transferred: 13.719 GBytes (25.974 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 9m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 42% /31.928G, 36.937M/s, 8m24s

2018/06/11 17:05:02 INFO : Google drive root ‘The Skull/Complete’: Modify window is 1ms
2018/06/11 17:05:02 INFO :
Transferred: 21.852 GBytes (26.607 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 14m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 68% /31.928G, 28.004M/s, 6m8s

2018/06/11 17:05:03 INFO :
Transferred: 18.812 GBytes (24.673 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 13m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 58% /31.928G, 28.493M/s, 7m51s

2018/06/11 17:05:03 INFO :
Transferred: 23.762 GBytes (26.996 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 15m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 74% /31.928G, 31.142M/s, 4m28s

2018/06/11 17:05:03 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:05:03 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:05:23 INFO : Google drive root ‘The Skull/Complete’: Waiting for checks to finish
2018/06/11 17:05:23 INFO : Google drive root ‘The Skull/Complete’: Waiting for transfers to finish
2018/06/11 17:05:23 INFO :
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 1m22.5s

2018/06/11 17:05:24 INFO : Google drive root ‘The Skull/Complete’: Waiting for checks to finish
2018/06/11 17:05:24 INFO : Google drive root ‘The Skull/Complete’: Waiting for transfers to finish
2018/06/11 17:05:24 INFO :
Transferred: 0 Bytes (0 Bytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 22.5s

2018/06/11 17:06:02 INFO :
Transferred: 29.250 GBytes (24.942 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 20m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 91% /31.928G, 27.390M/s, 1m40s

2018/06/11 17:06:02 INFO :
Transferred: 29.312 GBytes (26.310 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 19m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 91% /31.928G, 23.045M/s, 1m56s

2018/06/11 17:06:02 INFO :
Transferred: 29.311 GBytes (27.771 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 18m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 91% /31.928G, 22.737M/s, 1m57s

2018/06/11 17:06:02 INFO :
Transferred: 20.701 GBytes (27.133 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 13m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 64% /31.928G, 27.868M/s, 6m52s

2018/06/11 17:06:02 INFO : Google drive root ‘The Skull/Complete’: Modify window is 1ms
2018/06/11 17:06:02 INFO :
Transferred: 15.638 GBytes (26.652 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 10m0.8s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 48% /31.928G, 29.427M/s, 9m26s

2018/06/11 17:06:02 INFO :
Transferred: 23.375 GBytes (26.566 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 15m0.9s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 73% /31.928G, 23.653M/s, 6m10s

2018/06/11 17:06:03 INFO :
Transferred: 20.401 GBytes (24.847 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 14m0.7s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 63% /31.928G, 28.841M/s, 6m49s

2018/06/11 17:06:03 INFO :
Transferred: 25.391 GBytes (27.047 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 16m1.2s
Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 79% /31.928G, 25.068M/s, 4m27s

Transferring:

  • …p.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: 84% /31.928G, 27.605M/s, 3m1s
    2018/06/11 17:07:03 INFO : Google drive root ‘The Skull/Complete’: Modify window is 1ms
    2018/06/11 17:07:05 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
    2018/06/11 17:07:05 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
    2018/06/11 17:07:29 INFO : Google drive root ‘The Skull/Complete’: Waiting for checks to finish
    2018/06/11 17:07:29 INFO : Google drive root ‘The Skull/Complete’: Waiting for transfers to finish
    2018/06/11 17:07:29 INFO :
    Transferred: 0 Bytes (0 Bytes/s)
    Errors: 0
    Checks: 0
    Transferred: 0
    Elapsed time: 27.2s

2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Failed to read src hash: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Not deleting source as copy failed: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 ERROR : Attempt 1/3 failed with 1 errors and: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Failed to read src hash: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Not deleting source as copy failed: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 ERROR : Attempt 1/3 failed with 1 errors and: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Failed to read src hash: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Not deleting source as copy failed: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 ERROR : Attempt 1/3 failed with 1 errors and: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:52 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:53 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:53 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:53 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:53 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:53 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:54 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:54 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:54 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:54 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:07:54 NOTICE: 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Duplicate object found in destination - ignoring
2018/06/11 17:08:02 INFO :
Transferred: 31.928 GBytes (24.752 MBytes/s)
Errors: 0
Checks: 0
Transferred: 0
Elapsed time: 22m0.8s[/CODE]

2

Sorry! Don’t know why code tags don’t work on this log :frowning:

EDIT:

I just discovered it has created 5 copies of a 65 gig file during the night!

Arggghhh!

3

Are you copying bits of torrents that aren’t completed yet?

4

No, these are all completed files that were seeding.

It only appears to be the very large files that are affected, between 20 - 60 gigs in size.

Is it possible that Rclone is trying to pull them across multiple times because they haven’t finished appearing on GDrive?

I’m a bit confused about what’s going on, but it’s hugely frustrating as it’s triggering the ban

5

Did the file move after it was completed?

It seems to be moving while being copied:

2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Failed to read src hash: hash: failed to stat: lstat /media/dma/craftyclown/private/rtorrent/data/Complete/4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: no such file or directory
2018/06/11 17:07:52 INFO : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Copied (new)
2018/06/11 17:07:52 ERROR : 4K Film/Forrest.Gump.1994.2160p.UHD.BluRay.X265-IAMABLE.mkv: Not deleting source as copy failed: hash: failed to stat: lstat
6

Not that I’m aware of. One of the files that has been duped has been sat in my seedbox for months.

EDIT: I was deleting them as I saw the dupes appear via the GDrive web interface, if that’s what you mean?

Is there something odd about my script that might be at fault?

7

So I’m currently getting this if I try to run the script;

[pallas ~] /media/dma/craftyclown/rclone-cron.sh
runtime: failed to create new OS thread (have 34 already; errno=11)
runtime: may need to increase max user processes (ulimit -u)
fatal error: newosproc

runtime stack:
runtime.throw(0x1024803, 0x9)
/opt/go/go1.10/src/runtime/panic.go:619 +0x81
runtime.newosproc(0xc42055bc00, 0xc4206d4000)
/opt/go/go1.10/src/runtime/os_linux.go:164 +0x1af
runtime.newm1(0xc42055bc00)
/opt/go/go1.10/src/runtime/proc.go:1874 +0x113
runtime.newm(0x107efb8, 0x0)
/opt/go/go1.10/src/runtime/proc.go:1853 +0x9b
runtime.startTheWorldWithSema(0xc42040ed01, 0x459e23)
/opt/go/go1.10/src/runtime/proc.go:1150 +0x1d0
runtime.gcMarkTermination.func3()
/opt/go/go1.10/src/runtime/mgc.go:1647 +0x26
runtime.systemstack(0x0)
/opt/go/go1.10/src/runtime/asm_amd64.s:409 +0x79
runtime.mstart()
/opt/go/go1.10/src/runtime/proc.go:1170

goroutine 50 [running]:
runtime.systemstack_switch()
/opt/go/go1.10/src/runtime/asm_amd64.s:363 fp=0xc42040ed50 sp=0xc42040ed48 pc=0x456120
runtime.gcMarkTermination(0x3fede4aae6ffb9f4)
/opt/go/go1.10/src/runtime/mgc.go:1647 +0x407 fp=0xc42040ef20 sp=0xc42040ed50 pc=0x418f37
runtime.gcMarkDone()
/opt/go/go1.10/src/runtime/mgc.go:1513 +0x22c fp=0xc42040ef48 sp=0xc42040ef20 pc=0x418acc
runtime.gcBgMarkWorker(0xc42004e500)
/opt/go/go1.10/src/runtime/mgc.go:1912 +0x2e7 fp=0xc42040efd8 sp=0xc42040ef48 pc=0x419a47
runtime.goexit()
/opt/go/go1.10/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42040efe0 sp=0xc42040efd8 pc=0x458c41
created by runtime.gcBgMarkStartWorkers
/opt/go/go1.10/src/runtime/mgc.go:1723 +0x79

goroutine 1 [select]:
net/http.(*Transport).getConn(0xc42034ec30, 0xc42033af00, 0x0, 0xc42040c000, 0x5, 0xc420356d60, 0x16, 0x0, 0x0, 0x4)
/opt/go/go1.10/src/net/http/transport.go:962 +0x558
net/http.(*Transport).RoundTrip(0xc42034ec30, 0xc420362900, 0xa, 0xc4200ee400, 0xc)
/opt/go/go1.10/src/net/http/transport.go:409 +0x632
github.com/ncw/rclone/fs/fshttp.(*Transport).RoundTrip(0xc42033aa80, 0xc420362900, 0xc420362900, 0x0, 0xffffffffffffffff)
/home/ncw/go/src/github.com/ncw/rclone/fs/fshttp/http.go:285 +0x10b
github.com/ncw/rclone/vendor/golang.org/x/oauth2.(*Transport).RoundTrip(0xc42033ac00, 0xc420362800, 0xc42033ac00, 0x0, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/vendor/golang.org/x/oauth2/transport.go:48 +0xcf
net/http.send(0xc420362800, 0x112d320, 0xc42033ac00, 0x0, 0x0, 0x0, 0xc420344118, 0xf, 0xc42024b0f8, 0x1)
/opt/go/go1.10/src/net/http/client.go:252 +0x185
net/http.(*Client).send(0xc42033ac30, 0xc420362800, 0x0, 0x0, 0x0, 0xc420344118, 0x0, 0x1, 0x100)
/opt/go/go1.10/src/net/http/client.go:176 +0xfa
net/http.(*Client).Do(0xc42033ac30, 0xc420362800, 0x102a754, 0xf, 0x171be20)
/opt/go/go1.10/src/net/http/client.go:615 +0x28d
github.com/ncw/rclone/vendor/google.golang.org/api/gensupport.SendRequest(0x0, 0x0, 0xc42033ac30, 0xc420362800, 0x0, 0x0, 0xc420362800)
/home/ncw/go/src/github.com/ncw/rclone/vendor/google.golang.org/api/gensupport/send.go:43 +0x254
github.com/ncw/rclone/vendor/google.golang.org/api/drive/v3.(*FilesListCall).doRequest(0xc42035a440, 0x101edc4, 0x4, 0x0, 0x57, 0xc4203fe120)
/home/ncw/go/src/github.com/ncw/rclone/vendor/google.golang.org/api/drive/v3/drive-gen.go:5112 +0x37b
github.com/ncw/rclone/vendor/google.golang.org/api/drive/v3.(*FilesListCall).Do(0xc42035a440, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/vendor/google.golang.org/api/drive/v3/drive-gen.go:5124 +0x85
github.com/ncw/rclone/backend/drive.(*Fs).list.func1(0xc4203a8420, 0x16dc4e0, 0xc42024b580)
/home/ncw/go/src/github.com/ncw/rclone/backend/drive/drive.go:335 +0x98
github.com/ncw/rclone/lib/pacer.(*Pacer).call(0xc4203a8420, 0xc42033ad50, 0xa, 0xf758a0, 0xc4203fe101)
/home/ncw/go/src/github.com/ncw/rclone/lib/pacer/pacer.go:335 +0x171
github.com/ncw/rclone/lib/pacer.(*Pacer).Call(0xc4203a8420, 0xc42033ad50, 0xc42024b758, 0x1)
/home/ncw/go/src/github.com/ncw/rclone/lib/pacer/pacer.go:358 +0x66
github.com/ncw/rclone/backend/drive.(*Fs).list(0xc4203f2090, 0x101efa8, 0x4, 0x7ffc0fcd6839, 0x9, 0x1, 0xc4201c1858, 0xc42024b878, 0x7a520e, 0x7ffc0fcd6839)
/home/ncw/go/src/github.com/ncw/rclone/backend/drive/drive.go:334 +0x703
github.com/ncw/rclone/backend/drive.(*Fs).FindLeaf(0xc4203f2090, 0x101efa8, 0x4, 0x7ffc0fcd6839, 0x9, 0x0, 0x0, 0x0, 0xc42024b908, 0xc42024b908)
/home/ncw/go/src/github.com/ncw/rclone/backend/drive/drive.go:618 +0xb3
github.com/ncw/rclone/lib/dircache.(*DirCache)._findDir(0xc4203f21b0, 0x7ffc0fcd6839, 0x9, 0x0, 0x7ffc0fcd6843, 0x5, 0x102f090, 0x102f090)
/home/ncw/go/src/github.com/ncw/rclone/lib/dircache/dircache.go:181 +0x12f
github.com/ncw/rclone/lib/dircache.(*DirCache)._findDir(0xc4203f21b0, 0x7ffc0fcd6839, 0xf, 0xc4203f2000, 0xc42035a3f0, 0x7a4ae6, 0xc4203f21b0, 0xc4203f21b0)
/home/ncw/go/src/github.com/ncw/rclone/lib/dircache/dircache.go:174 +0xd0
github.com/ncw/rclone/lib/dircache.(*DirCache).FindRoot(0xc4203f21b0, 0x102f000, 0x0, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/lib/dircache/dircache.go:233 +0xa4
github.com/ncw/rclone/backend/drive.NewFs(0x7ffc0fcd6831, 0x6, 0x7ffc0fcd6838, 0x10, 0x6, 0x7ffc0fcd6838, 0x10, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/backend/drive/drive.go:555 +0x415
github.com/ncw/rclone/fs.NewFs(0x7ffc0fcd6831, 0x17, 0x88, 0x88, 0xc4203b1200, 0xc4203b4400)
/home/ncw/go/src/github.com/ncw/rclone/fs/fs.go:819 +0x9f
github.com/ncw/rclone/cmd.newFsDst(0x7ffc0fcd6831, 0x17, 0x113f360, 0xc42035e500)
/home/ncw/go/src/github.com/ncw/rclone/cmd/cmd.go:196 +0x4d
github.com/ncw/rclone/cmd.NewFsSrcDst(0xc4203680f0, 0x2, 0x5, 0xc4203680f0, 0x2, 0x5, 0xc4203b8340)
/home/ncw/go/src/github.com/ncw/rclone/cmd/cmd.go:207 +0x93
github.com/ncw/rclone/cmd/sync.glob..func1(0x16f0bc0, 0xc4203680f0, 0x2, 0x5)
/home/ncw/go/src/github.com/ncw/rclone/cmd/sync/sync.go:38 +0x8d
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).execute(0x16f0bc0, 0xc4203680a0, 0x5, 0x5, 0x16f0bc0, 0xc4203680a0)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:760 +0x2c1
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x16e8920, 0xd74dce, 0xc42024bf78, 0x4042ac)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:846 +0x30a
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).Execute(0x16e8920, 0xc42024bf58, 0x0)
/home/ncw/go/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:794 +0x2b
main.main()
/home/ncw/go/src/github.com/ncw/rclone/rclone.go:16 +0x31

goroutine 18 [syscall]:
os/signal.signal_recv(0x0)
/opt/go/go1.10/src/runtime/sigqueue.go:139 +0xa6
os/signal.loop()
/opt/go/go1.10/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
/opt/go/go1.10/src/os/signal/signal_unix.go:28 +0x41

goroutine 34 [chan receive]:
net/http.(*persistConn).addTLS(0xc4203ad200, 0xc420356d60, 0x12, 0x0, 0xc420356d73, 0x3)
/opt/go/go1.10/src/net/http/transport.go:1074 +0x1c8
net/http.(*Transport).dialConn(0xc42034ec30, 0x1138220, 0xc4200ee010, 0x0, 0xc42040c000, 0x5, 0xc420356d60, 0x16, 0x0, 0x0, …)
/opt/go/go1.10/src/net/http/transport.go:1153 +0x14c3
net/http.(*Transport).getConn.func4(0xc42034ec30, 0x1138220, 0xc4200ee010, 0xc42033af30, 0xc4203fe180)
/opt/go/go1.10/src/net/http/transport.go:957 +0x78
created by net/http.(*Transport).getConn
/opt/go/go1.10/src/net/http/transport.go:956 +0x363

goroutine 5 [runnable]:
reflect.typedslicecopy(0xe35000, 0xc4208dc080, 0x7, 0x7, 0xc4208ee050, 0x7, 0xa, 0x7f47f5664000)
/opt/go/go1.10/src/runtime/mbarrier.go:363 +0x108
reflect.Copy(0xec0e40, 0xc4208ec178, 0x197, 0xe22860, 0xc4208f2080, 0x97, 0x97)
/opt/go/go1.10/src/reflect/value.go:1934 +0x1b0
encoding/asn1.parseField(0xec0e40, 0xc4208ec178, 0x197, 0xc4208ea147, 0xd, 0x37b, 0x0, 0x0, 0x0, 0x0, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:825 +0x216a
encoding/asn1.parseField(0xefb2c0, 0xc4208ec178, 0x199, 0xc4208ea145, 0x120, 0x37d, 0x0, 0x0, 0x0, 0x0, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:913 +0x115f
encoding/asn1.parseField(0xf31fa0, 0xc4208ec160, 0x199, 0xc4208ea008, 0x3a5, 0x4ba, 0x139, 0x0, 0x0, 0x0, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:913 +0x115f
encoding/asn1.parseField(0xfda560, 0xc4208ec018, 0x199, 0xc4208ea004, 0x4bd, 0x4be, 0x0, 0x0, 0x0, 0x0, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:913 +0x115f
encoding/asn1.parseField(0xf5ef00, 0xc4208ec000, 0x199, 0xc4208ea000, 0x4c1, 0x4c2, 0x0, 0x0, 0x0, 0x0, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:913 +0x115f
encoding/asn1.UnmarshalWithParams(0xc4208ea000, 0x4c1, 0x4c2, 0xdf7a80, 0xc4208ec000, 0x0, 0x0, 0x410e88, 0x300, 0xf5ef00, …)
/opt/go/go1.10/src/encoding/asn1/asn1.go:1054 +0x14f
encoding/asn1.Unmarshal(0xc4208ea000, 0x4c1, 0x4c2, 0xdf7a80, 0xc4208ec000, 0xc4208e201c, 0xc4208e2697, 0xc4208e6000, 0xc4201bd440, 0xc4208e201c)
/opt/go/go1.10/src/encoding/asn1/asn1.go:1047 +0x68
crypto/x509.ParseCertificate(0xc4208ea000, 0x4c1, 0x4c2, 0xc4208980f0, 0xc4208e26a8, 0x0)
/opt/go/go1.10/src/crypto/x509/x509.go:1577 +0x74
crypto/x509.(*CertPool).AppendCertsFromPEM(0xc4203369c0, 0xc4208e2000, 0x6a8, 0x8a8, 0x8a8)
/opt/go/go1.10/src/crypto/x509/cert_pool.go:123 +0x102
crypto/x509.loadSystemRoots(0xc4201bd5f0, 0x10, 0xc4201bd5e8)
/opt/go/go1.10/src/crypto/x509/root_unix.go:74 +0x48c
crypto/x509.initSystemRoots()
/opt/go/go1.10/src/crypto/x509/root.go:21 +0x22
sync.(*Once).Do(0x171ab90, 0x107cc28)
/opt/go/go1.10/src/sync/once.go:44 +0xbe
crypto/x509.systemRootsPool(0x17)
/opt/go/go1.10/src/crypto/x509/root.go:16 +0x39
crypto/x509.(*Certificate).Verify(0xc42047a000, 0xc420356d60, 0x12, 0xc420336930, 0x0, 0xbec00cc24f8a7852, 0x888bd72, 0x16fc920, 0x0, 0x0, …)
/opt/go/go1.10/src/crypto/x509/verify.go:815 +0x734
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc4201bde70, 0xc420462160, 0x14e)
/opt/go/go1.10/src/crypto/tls/handshake_client.go:339 +0x526
crypto/tls.(*clientHandshakeState).handshake(0xc4201bde70, 0xc42045c000, 0x0)
/opt/go/go1.10/src/crypto/tls/handshake_client.go:247 +0x481
crypto/tls.(*Conn).clientHandshake(0xc42045a000, 0x107f1b0, 0xc42045a120)
/opt/go/go1.10/src/crypto/tls/handshake_client.go:168 +0x38c
crypto/tls.(*Conn).Handshake(0xc42045a000, 0x0, 0x0)
/opt/go/go1.10/src/crypto/tls/conn.go:1329 +0x185
net/http.(*persistConn).addTLS.func2(0x0, 0xc42045a000, 0xc420338140, 0xc4204500c0)
/opt/go/go1.10/src/net/http/transport.go:1068 +0x42
created by net/http.(*persistConn).addTLS
/opt/go/go1.10/src/net/http/transport.go:1064 +0x19f
[pallas ~]

8

Have I broken something?

9

Your script is running multiple times.

Run killall rclone-cron.sh to stop all the scripts and adjust your script

from

#!/bin/bash if pidof -o %PPID -x “rclone-cron.sh”; then exit 1 fi

to

#!/bin/bash
if pidof -o %PPID -x "rclone-cron.sh"; then exit 1 fi