Stuck on checking. Separate lsjson calls work fine!

What is the problem you are having with rclone?

I am trying to verify a sync (see this question) and whenever I do the sync, it gets totally stuck on "checking" a bunch of files

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.1
- os/version: darwin 10.13.6 (64 bit)
- os/kernel: 17.7.0 (x86_64)
- os/type: darwin
- os/arch: amd64
- go/version: go1.17.9
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

B2 to local

(It shouldn't matter but this is a copy of a crypt does outside of crypt. Hence the long names)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy -vv b2:jgwrclonetest/crypt . --progress --transfers 5 -vv --dry-run  --fast-list

The rclone config contents with secrets removed.

[b2]
type = b2
account = <REDACTED>
key = <REDACTED>

A log from the command with the -vv flag

The log is VERY LONG. It is mostly DEBUG messages of files it is skipping

2022/06/02 06:56:45 DEBUG : rclone: Version "v1.58.1" starting with parameters ["rclone" "copy" "-vv" "b2:jgwrclonetest/crypt" "." "--progress" "--transfers" "5" "-vv" "--dry-run" "--fast-list" "--log-file" "/Users/jwinokur/log"]
2022/06/02 06:56:45 DEBUG : Creating backend with remote "b2:jgwrclonetest/crypt"
2022/06/02 06:56:45 DEBUG : Using config file from "/Users/jwinokur/.config/rclone/rclone.conf"
2022/06/02 06:56:46 DEBUG : Couldn't decode error response: EOF
2022/06/02 06:56:46 DEBUG : Creating backend with remote "."
2022/06/02 06:56:46 DEBUG : fs cache: renaming cache item "." to be canonical "/Volumes/External8/backup/no_backblaze/nki1qaakmq30caqp4a12b6ikmk"

<snip...example:>
2022/06/02 06:56:47 DEBUG : 8lem2ulise948b67jvcaanhejk: Unchanged skipping
2022/06/02 06:56:47 DEBUG : q4sl05on346bm5bn5dup5osans: Size and modification time the same (differ by -252ms, within tolerance 1s)

I am cutting of nothing but lines like the example. If really needed, I can post the whole log but, to my knowledge, rclone doesn't show anything in the log while checking.

However, in the progress, it is stuck (5+min. no movement) on

Checking:
 * jsnpjbkfjhe0hjcp6a1e6b…5l2t3tbenu48grbmbc442g: checking
 * jsnpjbkfjhe0hjcp6a1e6b…ncgick1oje8vtk1n8basrm: checking
 * jsnpjbkfjhe0hjcp6a1e6b…sho2s590b9id8t3g9tfn70: checking
 * jsnpjbkfjhe0hjcp6a1e6b…lh29d5orekh51355snbgmo: checking
 * jsnpjbkfjhe0hjcp6a1e6b…rbpggghosvj0sqjje78hgg: checking
 * jsnpjbkfjhe0hjcp6a1e6b…vcing6g2ihgehi3g45tq2o: checking
 * jsnpjbkfjhe0hjcp6a1e6b…qij7p2qm9jsbhv5ovp15fo: checking
 * jsnpjbkfjhe0hjcp6a1e6b…tbplggftsf17q9vuicefg4: checking

(The Couldn't decode error response: EOF is a false flag. See @ncw's answer here)

Here's the interesting thing, I can run: (less than 30 sec)

$ rclone lsjson --files-only -R . |grep 5l2t3tbenu48grbmbc442g
{"Path":"jsnpjbkfjhe0hjcp6a1e6betf0/g0plsl5uhgj5448lgee2kmqivg7hc66k6bdc792ag0e7acma6q26jfj5l2t3tbenu48grbmbc442g","Name":"g0plsl5uhgj5448lgee2kmqivg7hc66k6bdc792ag0e7acma6q26jfj5l2t3tbenu48grbmbc442g","Size":1464816927,"MimeType":"application/octet-stream","ModTime":"2022-05-05T06:01:32-06:00","IsDir":false},

and (less than 10 sec)

$ rclone lsjson --files-only -R b2:jgwrclonetest/crypt |grep 5l2t3tbenu48grbmbc442g
{"Path":"jsnpjbkfjhe0hjcp6a1e6betf0/g0plsl5uhgj5448lgee2kmqivg7hc66k6bdc792ag0e7acma6q26jfj5l2t3tbenu48grbmbc442g","Name":"g0plsl5uhgj5448lgee2kmqivg7hc66k6bdc792ag0e7acma6q26jfj5l2t3tbenu48grbmbc442g","Size":1464816927,"MimeType":"application/octet-stream","ModTime":"2020-09-17T16:53:51.000Z","IsDir":false,"ID":"4_z074fa70789597d5a7a2d0913_f222e13388ba7e429_d20220505_m120132_c000_v0001069_t0007_u01651752092258"},

So rclone can see the files that way.

BTW, it works fine on --size-only

Thoughts? Thanks!

I'm having the same issue with gdrive. When I delete the --fast-list switch everything works, albeit really slowly, but it works. If I keep the --fast-list switch it seems like no progress is being made. It didn't used to be that way.

What you describe actually makes sense with --fast-list since it is getting everything before doing anything. What I am seeing is that it gets stuck after making tons of comparisons.

As I said, directly listing was very fast but just to check, I added the --fast-list flag to the lsjson example:

$ time rclone lsjson --files-only -R b2:jgwrclonetest/crypt >/dev/null 2>&1

real	0m1.317s
user	0m0.134s
sys	0m0.087s

$ time rclone lsjson --files-only -R b2:jgwrclonetest/crypt --fast-list >/dev/null 2>&1

real	0m1.133s
user	0m0.150s
sys	0m0.039s

Very little difference

@jwink3101 can you run rclone until it gets completely stuck, then kill -QUIT it or press CTRL-\. This will produce a vey long backtrace - can you post that please and I'll figure out where rclone is stuck - thank you.

It is below:

SIGQUIT: quit
PC=0x7fff76845a16 m=0 sigcode=0

goroutine 0 [idle]:
runtime.pthread_cond_wait(0x6b3bfa0, 0x6b3bf60)
	runtime/sys_darwin.go:444 +0x34
runtime.semasleep(0xffffffffffffffff)
	runtime/os_darwin.go:66 +0xad
runtime.notesleep(0x6b3bd50)
	runtime/lock_sema.go:182 +0x85
runtime.mPark()
	runtime/proc.go:1441 +0x2a
runtime.stopm()
	runtime/proc.go:2408 +0x78
runtime.findrunnable()
	runtime/proc.go:2984 +0x865
runtime.schedule()
	runtime/proc.go:3367 +0x239
runtime.park_m(0xc000408ea0)
	runtime/proc.go:3516 +0x14d
runtime.mcall()
	runtime/asm_amd64.s:307 +0x43

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x0)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0x57dfc20)
	sync/waitgroup.go:130 +0x71
github.com/rclone/rclone/fs/sync.(*syncCopyMove).stopCheckers(0xc0003a6000)
	github.com/rclone/rclone/fs/sync/sync.go:422 +0x6a
github.com/rclone/rclone/fs/sync.(*syncCopyMove).run(0xc0003a6000)
	github.com/rclone/rclone/fs/sync/sync.go:869 +0x2de
github.com/rclone/rclone/fs/sync.runSyncCopyMove({0x5c70828, 0xc000058118}, {0x5ca7028, 0xc000574510}, {0x5ca5f18, 0xc0007d2e00}, 0x0, 0x0, 0x86, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:1094 +0x21f
github.com/rclone/rclone/fs/sync.CopyDir(...)
	github.com/rclone/rclone/fs/sync/sync.go:1105
github.com/rclone/rclone/cmd/copy.glob..func1.1()
	github.com/rclone/rclone/cmd/copy/copy.go:86 +0x5b
github.com/rclone/rclone/cmd.Run(0x1, 0x1, 0x2, 0xc000e59d50)
	github.com/rclone/rclone/cmd/cmd.go:255 +0x11d
github.com/rclone/rclone/cmd/copy.glob..func1(0x6b203c0, {0xc000442000, 0x2, 0x9})
	github.com/rclone/rclone/cmd/copy/copy.go:84 +0xcf
github.com/spf13/cobra.(*Command).execute(0x6b203c0, {0xc000574630, 0x9, 0x9})
	github.com/spf13/cobra@v1.2.1/command.go:860 +0x5f8
github.com/spf13/cobra.(*Command).ExecuteC(0x6b1bb40)
	github.com/spf13/cobra@v1.2.1/command.go:974 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.2.1/command.go:902
github.com/rclone/rclone/cmd.Main()
	github.com/rclone/rclone/cmd/cmd.go:553 +0x76
main.main()
	github.com/rclone/rclone/rclone.go:14 +0x17

goroutine 19 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000147a00)
	go.opencensus.io@v0.23.0/stats/view/worker.go:276 +0xb9
created by go.opencensus.io/stats/view.init.0
	go.opencensus.io@v0.23.0/stats/view/worker.go:34 +0x92

goroutine 32 [select]:
github.com/rclone/rclone/cmd.startProgress.func3()
	github.com/rclone/rclone/cmd/progress.go:57 +0x10d
created by github.com/rclone/rclone/cmd.startProgress
	github.com/rclone/rclone/cmd/progress.go:49 +0x152

goroutine 36 [syscall, 1 minutes]:
os/signal.signal_recv()
	runtime/sigqueue.go:166 +0x28
os/signal.loop()
	os/signal/signal_unix.go:24 +0x19
created by os/signal.Notify.func1.1
	os/signal/signal.go:151 +0x2c

goroutine 37 [chan receive, 1 minutes]:
github.com/rclone/rclone/cmd.SigInfoHandler.func1()
	github.com/rclone/rclone/cmd/siginfo_darwin.go:20 +0x94
created by github.com/rclone/rclone/cmd.SigInfoHandler
	github.com/rclone/rclone/cmd/siginfo_darwin.go:19 +0xa5

goroutine 78 [runnable]:
syscall.syscall(0x4087300, 0x11, 0xc0009ca000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc00121c180, {0xc0009ca000, 0x0, 0x0})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc00121c180, {0xc0009ca000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00052e0f8, {0xc0009ca000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc000f2cde0}, {0x5c350c0, 0xc00052e0f8}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00052e0f8}, 0xc00052e0f8)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc000587560, {0x0, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 76 [runnable]:
syscall.syscall(0x4087300, 0xf, 0xc0009c2000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc00121c120, {0xc0009c2000, 0x0, 0x0})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc00121c120, {0xc0009c2000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00052e0f0, {0xc0009c2000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc000f2cd98}, {0x5c350c0, 0xc00052e0f0}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00052e0f0}, 0xc00052e0f0)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc0005874a0, {0x0, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 38 [semacquire]:
sync.runtime_Semacquire(0xc0000c2820)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc00056ca20)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc00089dd10)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc000467730}, {0x5ca7098, 0xc000e62120}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc000467730}, {0x5ca7098, 0xc000e62120})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc000467730}, {0x5ca7098, 0xc000e62120}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc000467730}, {0x5ca7098, 0xc000e62120})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc000e62120}, {0x5ca5f88, 0xc000467730})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 39 [semacquire]:
sync.runtime_Semacquire(0xc0000c2d00)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc000f2cd80)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc001025f50)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba310}, {0x5ca7098, 0xc0005874a0}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba310}, {0x5ca7098, 0xc0005874a0})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba310}, {0x5ca7098, 0xc0005874a0}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba310}, {0x5ca7098, 0xc0005874a0})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc0005874a0}, {0x5ca5f88, 0xc0008ba310})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 40 [semacquire]:
sync.runtime_Semacquire(0xc0000c2ea0)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc00000f818)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc00089d0b0)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba460}, {0x5ca7098, 0xc0005875c0}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba460}, {0x5ca7098, 0xc0005875c0})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba460}, {0x5ca7098, 0xc0005875c0}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba460}, {0x5ca7098, 0xc0005875c0})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc0005875c0}, {0x5ca5f88, 0xc0008ba460})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 41 [semacquire]:
sync.runtime_Semacquire(0xc0000c3040)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc000470d68)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc000efba40)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba230}, {0x5ca7098, 0xc0005873e0}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba230}, {0x5ca7098, 0xc0005873e0})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba230}, {0x5ca7098, 0xc0005873e0}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba230}, {0x5ca7098, 0xc0005873e0})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc0005873e0}, {0x5ca5f88, 0xc0008ba230})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 42 [semacquire]:
sync.runtime_Semacquire(0xc0000c31e0)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc000f2cdc8)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc001025fb0)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba3f0}, {0x5ca7098, 0xc000587560}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba3f0}, {0x5ca7098, 0xc000587560})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba3f0}, {0x5ca7098, 0xc000587560}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba3f0}, {0x5ca7098, 0xc000587560})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc000587560}, {0x5ca5f88, 0xc0008ba3f0})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 43 [semacquire]:
sync.runtime_Semacquire(0xc0000c3380)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc00000f8c0)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc00089d1a0)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba540}, {0x5ca7098, 0xc000587680}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba540}, {0x5ca7098, 0xc000587680})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba540}, {0x5ca7098, 0xc000587680}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba540}, {0x5ca7098, 0xc000587680})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc000587680}, {0x5ca5f88, 0xc0008ba540})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 44 [semacquire]:
sync.runtime_Semacquire(0xc0000c3520)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc00000f860)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc00089d110)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba4d0}, {0x5ca7098, 0xc000587620}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba4d0}, {0x5ca7098, 0xc000587620})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba4d0}, {0x5ca7098, 0xc000587620}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba4d0}, {0x5ca7098, 0xc000587620})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc000587620}, {0x5ca5f88, 0xc0008ba4d0})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 45 [semacquire]:
sync.runtime_Semacquire(0xc0000c36c0)
	runtime/sema.go:56 +0x25
sync.(*WaitGroup).Wait(0xc000899980)
	sync/waitgroup.go:130 +0x71
golang.org/x/sync/errgroup.(*Group).Wait(0xc000579380)
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:40 +0x27
github.com/rclone/rclone/fs/operations.checkHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba2a0}, {0x5ca7098, 0xc000587440}, 0x2)
	github.com/rclone/rclone/fs/operations/operations.go:86 +0x255
github.com/rclone/rclone/fs/operations.CheckHashes({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba2a0}, {0x5ca7098, 0xc000587440})
	github.com/rclone/rclone/fs/operations/operations.go:61 +0x15f
github.com/rclone/rclone/fs/operations.equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba2a0}, {0x5ca7098, 0xc000587440}, {0x0, 0x0, 0x1, 0x0})
	github.com/rclone/rclone/fs/operations/operations.go:221 +0x7ab
github.com/rclone/rclone/fs/operations.Equal({0x5c707f0, 0xc000980000}, {0x5c8cd50, 0xc0008ba2a0}, {0x5ca7098, 0xc000587440})
	github.com/rclone/rclone/fs/operations/operations.go:124 +0x85
github.com/rclone/rclone/fs/operations.NeedTransfer({0x5c707f0, 0xc000980000}, {0x5ca7098, 0xc000587440}, {0x5ca5f88, 0xc0008ba2a0})
	github.com/rclone/rclone/fs/operations/operations.go:1650 +0x5a5
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairChecker(0xc0003a6000, 0x0, 0x0, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:325 +0x227
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startCheckers
	github.com/rclone/rclone/fs/sync/sync.go:414 +0x47

goroutine 46 [select]:
github.com/rclone/rclone/fs/sync.(*pipe).GetMax(0xc0006a3540, {0x5c707f0, 0xc000980040}, 0x0)
	github.com/rclone/rclone/fs/sync/pipe.go:124 +0xa8
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0003a6000, {0x5c707f0, 0xc000980000}, 0x0, {0x5ca7028, 0xc000574510}, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:395 +0xe5
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:430 +0x47

goroutine 47 [select, 1 minutes]:
github.com/rclone/rclone/fs/sync.(*pipe).GetMax(0xc0006a3540, {0x5c707f0, 0xc000980040}, 0x14)
	github.com/rclone/rclone/fs/sync/pipe.go:124 +0xa8
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0003a6000, {0x5c707f0, 0xc000980000}, 0x0, {0x5ca7028, 0xc000574510}, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:395 +0xe5
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:430 +0x47

goroutine 48 [select, 1 minutes]:
github.com/rclone/rclone/fs/sync.(*pipe).GetMax(0xc0006a3540, {0x5c707f0, 0xc000980040}, 0x28)
	github.com/rclone/rclone/fs/sync/pipe.go:124 +0xa8
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0003a6000, {0x5c707f0, 0xc000980000}, 0x0, {0x5ca7028, 0xc000574510}, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:395 +0xe5
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:430 +0x47

goroutine 49 [select, 1 minutes]:
github.com/rclone/rclone/fs/sync.(*pipe).GetMax(0xc0006a3540, {0x5c707f0, 0xc000980040}, 0x3c)
	github.com/rclone/rclone/fs/sync/pipe.go:124 +0xa8
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0003a6000, {0x5c707f0, 0xc000980000}, 0x0, {0x5ca7028, 0xc000574510}, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:395 +0xe5
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:430 +0x47

goroutine 82 [select, 1 minutes]:
github.com/rclone/rclone/fs/sync.(*pipe).GetMax(0xc0006a3540, {0x5c707f0, 0xc000980040}, 0x50)
	github.com/rclone/rclone/fs/sync/pipe.go:124 +0xa8
github.com/rclone/rclone/fs/sync.(*syncCopyMove).pairCopyOrMove(0xc0003a6000, {0x5c707f0, 0xc000980000}, 0x0, {0x5ca7028, 0xc000574510}, 0x0, 0x0)
	github.com/rclone/rclone/fs/sync/sync.go:395 +0xe5
created by github.com/rclone/rclone/fs/sync.(*syncCopyMove).startTransfers
	github.com/rclone/rclone/fs/sync/sync.go:430 +0x47

goroutine 168 [runnable]:
syscall.syscall(0x4087300, 0xe, 0xc000eca000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc000ea63c0, {0xc000eca000, 0x4396700, 0xc00089d0b8})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc000ea63c0, {0xc000eca000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00023f5c0, {0xc000eca000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc00000f830}, {0x5c350c0, 0xc00023f5c0}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00023f5c0}, 0xc00023f5c0)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc0005875c0, {0x0, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 166 [runnable]:
syscall.syscall(0x4087300, 0xd, 0xc000ca6000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc00121dce0, {0xc000ca6000, 0x0, 0x0})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc00121dce0, {0xc000ca6000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00023f610, {0xc000ca6000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc00056ca38}, {0x5c350c0, 0xc00023f610}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00023f610}, 0xc00023f610)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc000e62120, {0x5535820, 0x6b6ba88}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 85 [runnable]:
syscall.syscall(0x4087300, 0x13, 0xc00106c000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc000e62660, {0xc00106c000, 0x6236653161367063, 0x7030672f30667465})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc000e62660, {0xc00106c000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc000514028, {0xc00106c000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc000470d80}, {0x5c350c0, 0xc000514028}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc000514028}, 0xc000514028)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc0005873e0, {0x5c707f0, 0xc00047bc00}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 60 [chan receive, 1 minutes]:
github.com/rclone/rclone/fs/march.(*March).Run.func2()
	github.com/rclone/rclone/fs/march/march.go:208 +0x45
created by github.com/rclone/rclone/fs/march.(*March).Run
	github.com/rclone/rclone/fs/march/march.go:206 +0x3a5

goroutine 170 [runnable]:
syscall.syscall(0x4087300, 0x10, 0xc000ed2000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc000ea6420, {0xc000ed2000, 0xd8b54a8, 0x87})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc000ea6420, {0xc000ed2000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00023f5e0, {0xc000ed2000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc00000f878}, {0x5c350c0, 0xc00023f5e0}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00023f5e0}, 0xc00023f5e0)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc000587620, {0x0, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 151 [runnable]:
syscall.syscall(0x4087300, 0xb, 0xc000d1c000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc0001f8780, {0xc000d1c000, 0xd988e48, 0x87})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc0001f8780, {0xc000d1c000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc000010588, {0xc000d1c000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc000899998}, {0x5c350c0, 0xc000010588}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc000010588}, 0xc000010588)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc000587440, {0x0, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 172 [runnable]:
syscall.syscall(0x4087300, 0x12, 0xc000eda000, 0x8000)
	runtime/sys_darwin.go:22 +0x3b
syscall.read(0xc000ea6480, {0xc000eda000, 0xda0d7b0, 0x87})
	syscall/zsyscall_darwin_amd64.go:1171 +0x49
syscall.Read(...)
	syscall/syscall_unix.go:189
internal/poll.ignoringEINTRIO(...)
	internal/poll/fd_unix.go:582
internal/poll.(*FD).Read(0xc000ea6480, {0xc000eda000, 0x8000, 0x8000})
	internal/poll/fd_unix.go:163 +0x285
os.(*File).read(...)
	os/file_posix.go:32
os.(*File).Read(0xc00023f608, {0xc000eda000, 0x8000, 0x8000})
	os/file.go:119 +0x5e
io.copyBuffer({0x5c34a80, 0xc00000f8d8}, {0x5c350c0, 0xc00023f608}, {0x0, 0x0, 0x0})
	io/io.go:423 +0x1b2
io.Copy(...)
	io/io.go:382
github.com/rclone/rclone/fs/hash.StreamTypes({0x5c350c0, 0xc00023f608}, 0xc00023f608)
	github.com/rclone/rclone/fs/hash/hash.go:118 +0x70
github.com/rclone/rclone/backend/local.(*Object).Hash(0xc000587680, {0x5ca5f00, 0x0}, 0x2)
	github.com/rclone/rclone/backend/local/local.go:906 +0x4ef
github.com/rclone/rclone/fs/operations.checkHashes.func2()
	github.com/rclone/rclone/fs/operations/operations.go:79 +0x45
golang.org/x/sync/errgroup.(*Group).Go.func1()
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x67
created by golang.org/x/sync/errgroup.(*Group).Go
	golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54 +0x92

goroutine 145 [select]:
github.com/rclone/rclone/fs/accounting.(*StatsInfo).averageLoop(0xc000103520)
	github.com/rclone/rclone/fs/accounting/stats.go:312 +0x16a
created by github.com/rclone/rclone/fs/accounting.(*StatsInfo).startAverageLoop.func1
	github.com/rclone/rclone/fs/accounting/stats.go:341 +0x72

rax    0x104
rbx    0x52b00
rcx    0x7ffeefbfeeb8
rdx    0x52b00
rdi    0x6b3bfa0
rsi    0x52b0100052c00
rbp    0x7ffeefbfef50
rsp    0x7ffeefbfeeb8
r8     0x0
r9     0x60
r10    0x0
r11    0x202
r12    0x6b3bfa0
r13    0x7ffeefbfeed8
r14    0x1
r15    0x7fffaecc1380
rip    0x7fff76845a16
rflags 0x203
cs     0x7
fs     0x0
gs     0x0

Thanks!

Is it possible that it was trying to locally hash the files and what I thought was stuck was just super slow hashing? Still a bug but a different take on the issue.

Yes, that looks like you've got 8 hashers doing their stuff.

How big are these files?

If you aren't on SSD you may benefit in reducing --checkers so your disk does a bit less seeking.

Why would they be hashing on a sync from local to B2 without the checksum flag?

It is spinning rust on 2 gb files plus USB2 plus 13 year old computer but I am still confused as to why it would fall back to hashing? And why just some of the files? Let me know if you need anything more!

If the size matches but the modtime doesn't, rclone will hash both the files and if they are the
same it will update the modtimes of the local files.

You can see this in the debug (-vv) log.