Cannot setup Google Drive on rClone MacOS

eli5 · May 15, 2019, 9:17am

Actions:
Attempting to setup a a Google Drive remote after a fresh rClone install. Using custom oAuth keys. Have used this Google Drive account with rClone (and other API stuff) in the past so confident keys are fine. Error appears via auto and headless authentication.

Error received:

Failed to configure token: failed to get token: Post https://accounts.google.com/o/oauth2/token: net/http: TLS handshake timeout

Efforts:

Changed networks
Reboot computer
Scour the interwebz for potential solutions
Prefix rclone config with sudo
Thought this "new" scope thing with Google's APIs are he cause so even ensured I checked all boxes for complete access despite verification not required for my needs [more of a sidenote here]

I eventually ran rclone config as rclone config -vv yet skipped the advanced config steps whilst testing. Below is the output:

If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
2019/05/15 18:57:33 DEBUG : Starting auth server on 127.0.0.1:53682
2019/05/15 18:57:53 DEBUG : Received request on auth server
2019/05/15 18:57:53 DEBUG : Successfully got code
Got code
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x312c00 pc=0x7fff9235a0dd]

runtime stack:
runtime.throw(0x1fb3685, 0x2a)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/panic.go:617 +0x72
runtime.sigpanic()
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/signal_unix.go:374 +0x4a9

goroutine 73 [syscall]:
runtime.cgocall(0x1001bd0, 0xc000233470, 0xc00042a7a8)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/cgocall.go:128 +0x5b fp=0xc000233440 sp=0xc000233408 pc=0x100503b
crypto/x509._Cfunc_FetchPEMRoots(0xc00042a7a0, 0xc00042a7a8, 0xc000216b00, 0x0)
	_cgo_gotypes.go:110 +0x4d fp=0xc000233470 sp=0xc000233440 pc=0x121c94d
crypto/x509.loadSystemRoots.func1(0xc00042a7a0, 0xc00042a7a8, 0x1b)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/x509/root_cgo_darwin.go:281 +0x12d fp=0xc0002334b0 sp=0xc000233470 pc=0x1220a7d
crypto/x509.loadSystemRoots(0x0, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/x509/root_cgo_darwin.go:281 +0xec fp=0xc000233580 sp=0xc0002334b0 pc=0x121cabc
crypto/x509.initSystemRoots()
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/x509/root.go:21 +0x26 fp=0xc0002335a8 sp=0xc000233580 pc=0x120f5d6
sync.(*Once).Do(0x2cb7788, 0x1ff5eb8)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/sync/once.go:44 +0xb3 fp=0xc0002335d8 sp=0xc0002335a8 pc=0x1068463
crypto/x509.systemRootsPool(...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/x509/root.go:16
crypto/x509.(*Certificate).Verify(0xc000124000, 0xc0002823e0, 0x13, 0xc000165440, 0x0, 0xbf2f1304550028e0, 0xce7186bf6, 0x2c9a400, 0x0, 0x0, ...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/x509/verify.go:744 +0x6ca fp=0xc0002336f0 sp=0xc0002335d8 pc=0x1213a5a
crypto/tls.(*Conn).verifyServerCertificate(0xc0002fc700, 0xc000164f60, 0x2, 0x2, 0x91c, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/tls/handshake_client.go:838 +0x26a fp=0xc000233888 sp=0xc0002336f0 pc=0x1245a0a
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc000233dc8, 0xc0000f26e0, 0x157)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/tls/handshake_client.go:454 +0x1869 fp=0xc000233bc0 sp=0xc000233888 pc=0x1243b59
crypto/tls.(*clientHandshakeState).handshake(0xc000233dc8, 0xc000186900, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/tls/handshake_client.go:399 +0x3fb fp=0xc000233cc8 sp=0xc000233bc0 pc=0x1241f1b
crypto/tls.(*Conn).clientHandshake(0xc0002fc700, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/tls/handshake_client.go:208 +0x2cd fp=0xc000233f38 sp=0xc000233cc8 pc=0x124056d
crypto/tls.(*Conn).Handshake(0xc0002fc700, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/crypto/tls/conn.go:1343 +0xef fp=0xc000233f78 sp=0xc000233f38 pc=0x123eb7f
net/http.(*persistConn).addTLS.func2(0x0, 0xc0002fc700, 0xc0000ca280, 0xc00009eea0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1190 +0x42 fp=0xc000233fc0 sp=0xc000233f78 pc=0x1328492
runtime.goexit()
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc000233fc8 sp=0xc000233fc0 pc=0x105e8b1
created by net/http.(*persistConn).addTLS
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1186 +0x1ab

goroutine 1 [select]:
net/http.(*Transport).getConn(0x2c83260, 0xc000164e10, 0x0, 0x1fb3145, 0x5, 0xc0002823e0, 0x17, 0x0, 0x0, 0x0, ...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1020 +0x63d
net/http.(*Transport).roundTrip(0x2c83260, 0xc000148200, 0x10, 0xc000206350, 0xc0004bb1f8)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:467 +0x6ef
net/http.(*Transport).RoundTrip(0x2c83260, 0xc000148200, 0x2c83260, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/roundtrip.go:17 +0x35
net/http.send(0xc000148200, 0x21e81e0, 0x2c83260, 0x0, 0x0, 0x0, 0xc0000ce108, 0x1fb3160, 0x1, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/client.go:250 +0x461
net/http.(*Client).send(0x2c9a080, 0xc000148200, 0x0, 0x0, 0x0, 0xc0000ce108, 0x0, 0x1, 0xc0004bb4a8)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/client.go:174 +0xfb
net/http.(*Client).do(0x2c9a080, 0xc000148200, 0x0, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/client.go:641 +0x279
net/http.(*Client).Do(...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/client.go:509
github.com/ncw/rclone/vendor/golang.org/x/net/context/ctxhttp.Do(0x2209de0, 0xc00003a0b0, 0x2c9a080, 0xc000148100, 0xc0001c8348, 0xc00015f5e0, 0xc000148100)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/golang.org/x/net/context/ctxhttp/ctxhttp.go:27 +0xd9
github.com/ncw/rclone/vendor/golang.org/x/oauth2/internal.RetrieveToken(0x2209de0, 0xc00003a0b0, 0xc0001f2140, 0x48, 0xc00018c080, 0x18, 0x1fb3145, 0x2a, 0xc000164c60, 0x0, ...)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/golang.org/x/oauth2/internal/token.go:204 +0x282
github.com/ncw/rclone/vendor/golang.org/x/oauth2.retrieveToken(0x2209de0, 0xc00003a0b0, 0xc0001c0850, 0xc000164c60, 0xc0001c8218, 0x9, 0x200)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/golang.org/x/oauth2/token.go:157 +0x86
github.com/ncw/rclone/vendor/golang.org/x/oauth2.(*Config).Exchange(0xc0001c0850, 0x2209de0, 0xc00003a0b0, 0xc0001820f2, 0x59, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/golang.org/x/oauth2/oauth2.go:203 +0x208
github.com/ncw/rclone/lib/oauthutil.doConfig(0x1f7d58b, 0x5, 0xc0001600f9, 0x4, 0x21f7da0, 0xc0001643f0, 0x0, 0xc0001c0850, 0x1, 0x0, ...)
	/Users/travis/gopath/src/github.com/ncw/rclone/lib/oauthutil/oauthutil.go:477 +0x874
github.com/ncw/rclone/lib/oauthutil.Config(...)
	/Users/travis/gopath/src/github.com/ncw/rclone/lib/oauthutil/oauthutil.go:344
github.com/ncw/rclone/backend/drive.init.0.func1(0xc0001600f9, 0x4, 0x21f7da0, 0xc0001643f0)
	/Users/travis/gopath/src/github.com/ncw/rclone/backend/drive/drive.go:177 +0x2e1
github.com/ncw/rclone/fs/config.RemoteConfig(0xc0001600f9, 0x4)
	/Users/travis/gopath/src/github.com/ncw/rclone/fs/config/config.go:810 +0xe1
github.com/ncw/rclone/fs/config.NewRemote(0xc0001600f9, 0x4)
	/Users/travis/gopath/src/github.com/ncw/rclone/fs/config/config.go:1084 +0x205
github.com/ncw/rclone/fs/config.EditConfig()
	/Users/travis/gopath/src/github.com/ncw/rclone/fs/config/config.go:1184 +0x236
github.com/ncw/rclone/cmd/config.glob..func1(0x2c864c0, 0xc0002ca0d0, 0x0, 0x1)
	/Users/travis/gopath/src/github.com/ncw/rclone/cmd/config/config.go:34 +0x56
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).execute(0x2c864c0, 0xc0002ca080, 0x1, 0x1, 0x2c864c0, 0xc0002ca080)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:766 +0x2ae
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x2c84f60, 0xc0003bff88, 0x100793f, 0xc0000a8058)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:852 +0x2ec
github.com/ncw/rclone/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/Users/travis/gopath/src/github.com/ncw/rclone/vendor/github.com/spf13/cobra/command.go:800
github.com/ncw/rclone/cmd.Main()
	/Users/travis/gopath/src/github.com/ncw/rclone/cmd/cmd.go:488 +0x47
main.main()
	/Users/travis/gopath/src/github.com/ncw/rclone/rclone.go:13 +0x20

goroutine 4 [syscall]:
os/signal.signal_recv(0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/sigqueue.go:139 +0x9f
os/signal.loop()
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/os/signal/signal_unix.go:29 +0x41

goroutine 66 [IO wait]:
internal/poll.runtime_pollWait(0x3419dd8, 0x72, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000174098, 0x72, 0x0, 0x0, 0x1f7e439)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Accept(0xc000174080, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/internal/poll/fd_unix.go:384 +0x1ba
net.(*netFD).accept(0xc000174080, 0xc98af7ca, 0x326c98af7ca, 0x100000001)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/fd_unix.go:238 +0x42
net.(*TCPListener).accept(0xc000168018, 0x5cdbd491, 0xc000074e28, 0x10bf906)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/tcpsock_posix.go:139 +0x32
net.(*TCPListener).Accept(0xc000168018, 0xc000074e78, 0x18, 0xc000482900, 0x1309b44)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/tcpsock.go:260 +0x48
net/http.(*Server).Serve(0xc00024a820, 0x2201ee0, 0xc000168018, 0x0, 0x0)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/server.go:2859 +0x22d
github.com/ncw/rclone/lib/oauthutil.(*authServer).Start(0xc00009e300)
	/Users/travis/gopath/src/github.com/ncw/rclone/lib/oauthutil/oauthutil.go:590 +0x34c
created by github.com/ncw/rclone/lib/oauthutil.doConfig
	/Users/travis/gopath/src/github.com/ncw/rclone/lib/oauthutil/oauthutil.go:448 +0xc0b

goroutine 51 [chan receive]:
net/http.(*persistConn).addTLS(0xc0001be6c0, 0xc0002823e0, 0x13, 0x0, 0xc0002823f4, 0x3)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1196 +0x1d4
net/http.(*Transport).dialConn(0x2c83260, 0x2209de0, 0xc00003a0b0, 0x0, 0x1fb3145, 0x5, 0xc0002823e0, 0x17, 0x0, 0xc000486780, ...)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1269 +0x1cde
net/http.(*Transport).getConn.func4(0x2c83260, 0x2209de0, 0xc00003a0b0, 0xc000164e40, 0xc000208240)
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1015 +0xa6
created by net/http.(*Transport).getConn
	/Users/travis/.gimme/versions/go1.12.4.darwin.amd64/src/net/http/transport.go:1014 +0x455

rClone Version:
rclone v1.47.0
os/arch: darwin/amd64
go version: go1.12.4

Mac Version:
10.10.5, Yosemite (14F2511)

I'm honestly stumped here and interpreting this stacktrace(?) is beyond my current expertise. Thanks in advance for your time.

ncw · May 15, 2019, 10:08am

Hmmm, this could be related to this go issue

github.com/golang/go

Issue: crypto/x509: FetchPEMRoots on macOS takes 20 seconds when there are a lot of trust settings

opened by tbodt on 2017-03-15

Repro steps
First run this test program:
package main
import (
 "encoding/pem"
 "crypto/x509"
)
func main() {
 const certPEM = `
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgIIE31FZVaPXTUwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
cm5ldCBBdXRob3JpdHkgRzIwHhcNMTQwMTI5MTMyNzQzWhcNMTQwNTI5MDAwMDAw
WjBpMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEYMBYGA1UEAwwPbWFp
bC5nb29nbGUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEfRrObuSW5T7q
5CnSEqefEmtH4CCv6+5EckuriNr1CjfVvqzwfAhopXkLrq45EQm8vkmf7W96XJhC
7ZM0dYi1/qOCAU8wggFLMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAa
BgNVHREEEzARgg9tYWlsLmdvb2dsZS5jb20wCwYDVR0PBAQDAgeAMGgGCCsGAQUF
BwEBBFwwWjArBggrBgEFBQcwAoYfaHR0cDovL3BraS5nb29nbGUuY29tL0dJQUcy
LmNydDArBggrBgEFBQcwAYYfaHR0cDovL2NsaWVudHMxLmdvb2dsZS5jb20vb2Nz
cDAdBgNVHQ4EFgQUiJxtimAuTfwb+aUtBn5UYKreKvMwDAYDVR0TAQH/BAIwADAf
BgNVHSMEGDAWgBRK3QYWG7z2aLV29YG2u2IaulqBLzAXBgNVHSAEEDAOMAwGCisG
AQQB1nkCBQEwMAYDVR0fBCkwJzAloCOgIYYfaHR0cDovL3BraS5nb29nbGUuY29t
L0dJQUcyLmNybDANBgkqhkiG9w0BAQUFAAOCAQEAH6RYHxHdcGpMpFE3oxDoFnP+
gtuBCHan2yE2GRbJ2Cw8Lw0MmuKqHlf9RSeYfd3BXeKkj1qO6TVKwCh+0HdZk283
TZZyzmEOyclm3UGFYe82P/iDFt+CeQ3NpmBg+GoaVCuWAARJN/KfglbLyyYygcQq
0SgeDh8dRKUiaW3HQSoYvTvdTuqzwK4CXsr3b5/dAOY8uMuG/IAR3FgwTbZ1dtoW
RvOTa8hYiU6A475WuZKyEHcwnGYe57u2I2KbMgcKjPniocj4QzgYsVAVKW3IwaOh
yE+vPxsiUkvQHdO2fojCkY8jg70jxM+gu59tPDNbw3Uh/2Ij310FgTHsnGQMyA==
-----END CERTIFICATE-----`
 block, _ :=...

NeedsInvestigation OS-Darwin help wanted

Have you got a lot of certificates installed?

That traceback is part of establishing a TLS connection when go calls the macOS C libraries which is in the same area.

If you do this (from the go issue) it will count the number of certificates you have.

security trust-settings-export -d trust.plist
grep '<integer>' trust.plist  | wc -l

eli5 · May 15, 2019, 5:50pm

Interesting, I only installed from the binary, thus Go is not on my system.

A quick scour before bed has me find some potentially helpful information. Well, not to directly to myself since it's Go related:

GoLang issue?

Go’s http package doesn’t specify request timeouts by default, allowing services to hijack your goroutines.
This article delves into the apparently known issue of Go.

A discrepancy between TLS versions of Go and the server?

Earlier today we permanently removed support for the following weak cryptographic standards on github(dot)com and api(dot)github(dot)com. (GitHub Announcement: https(collon)//github(dot)blog/2018-02-23-weak-cryptographic-standards-removed/)

[Google's TLS] facilitates API interactions with Google over TLS, including Google Cloud. TLS1.1-3 are supported (Whitepaper: https(collon)//cloud(dot)google(dot)com/security/encryption-in-transit/)

The net/http occurs when there is a large amount of http.Get() requests from client side. (Discussion includes sample code)

Issue with Keychain?

The Issue linked touches on a few things about trust policies and root CAs causing issues for some people (I think?). I have yet to make random amendments since messing with Keychain is also beyond my depth.

Hope the aforementioned references are helpful. P.S Excuse the poor link formatting. Needed to bypass the two-link limit

Have you got a lot of certificates installed?
I don't know, sorry. How can I check that? I do not have Go installed on this Mac. I neither recall recently playing with certificates or making heavy environment modifications to this machine.

ncw · May 16, 2019, 8:40am

Can you do this in a terminal window?

security trust-settings-export -d trust.plist grep '<integer>' trust.plist | wc -l
grep '<integer>' trust.plist  | wc -l

That will tell you how many certificates you have installed.

eli5 · May 16, 2019, 3:20pm

A copy-paste appears messy for this output so here's an image:

Doesn't appear any are showing? Unless I've done something wrong with the commands here.

Animosity022 · May 16, 2019, 4:01pm

I have a complete fresh mac install and installed rclone on latest Mojave and I had to install nothing else to run rclone. You shouldn't need go nor anything extra.

What's the version of Mac you are running on?

ncw · May 16, 2019, 4:14pm

Sorry I messed up the cut and paste, the two commands should look like this

security trust-settings-export -d trust.plist
grep '<integer>' trust.plist | wc -l

eli5 · May 18, 2019, 5:04pm

Not a problem. Below are the outputs of both commands:

~ > security trust-settings-export -d trust.plist
...Trust Settings exported successfully.
~ > grep '' trust.plist | wc -l
1

eli5 · May 18, 2019, 5:05pm

Yeah that's what I think too.

rClone Version:
rclone v1.47.0
os/arch: darwin/amd64
go version: go1.12.4

Mac Version:
10.10.5, Yosemite (14F2511)

Animosity022 · May 18, 2019, 6:13pm

I'm on the latest of Mojave so definitely a newer version.

ncw · May 19, 2019, 11:27am

Ah, so that means it isn't 100s of certificates causing the problem.

eli5 · May 19, 2019, 2:08pm

I'm sorry, what does this mean?

ncw · May 19, 2019, 6:44pm

It means that the go bug I linked above is not the cause of your problem.

eli5 · May 22, 2019, 6:31am

OK, great, yet can be the cause of the problem? I'm pretty keen to get rclone going.

ncw · May 22, 2019, 8:55am

Can you check firewalling? The error might be caused by rclone not being able to contact google's servers.

system · August 20, 2019, 8:55am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.