ncw
September 14, 2020, 3:06pm
10
I can see what is happening there...
Rclone issues the DELETE on the directory
2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : DELETE /v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Accept-Encoding: gzip
The server responds with a redirect
2020/09/12 22:44:36 DEBUG : HTTP RESPONSE (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : HTTP/1.1 302 Found
Cache-Control: private
Client-Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Content-Type: text/plain
Date: Sun, 13 Sep 2020 02:44:35 GMT
Location: https://redacted-my.sharepoint.com/_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Strict-Transport-Security: max-age=31536000
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"North Central US","Slice":"SliceC","Ring":"3","ScaleUnit":"003","RoleInstance":"AGSFE_IN_30"}}
Content-Length: 0
So rclone does the request at that URL
2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc0004be400)
2020/09/12 22:44:36 DEBUG : GET /_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: redacted-my.sharepoint.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Referer: https://graph.microsoft.com/v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Accept-Encoding: gzip
However it has turned the DELETE into a GET request...
After a lot of thinking, I think this is actually a bug in the go runtime and I've reported that here.
opened 03:02PM - 14 Sep 20 UTC
closed 11:36PM - 03 Dec 24 UTC
NeedsDecision
### What version of Go are you using (`go version`)?
<pre>
$ go version
go … version go1.15 linux/amd64
</pre>
### Does this issue reproduce with the latest release?
Yes
### What operating system and processor architecture are you using (`go env`)?
<details><summary><code>go env</code> Output</summary><br><pre>
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ncw/.cache/go-build"
GOENV="/home/ncw/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ncw/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ncw/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/go1.15"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/go1.15/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build061961387=/tmp/go-build -gno-record-gcc-switches"
</pre></details>
### What did you do?
We used http.Client against a Microsoft Ondrive service.
When we issued a DELETE response the server responded with a 302 redirect and http.Client retried with a GET request.
We issue a DELETE request
```
2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : DELETE /v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Accept-Encoding: gzip
```
The server responds with a 302 redirect
```
2020/09/12 22:44:36 DEBUG : HTTP RESPONSE (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : HTTP/1.1 302 Found
Cache-Control: private
Client-Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Content-Type: text/plain
Date: Sun, 13 Sep 2020 02:44:35 GMT
Location: https://redacted-my.sharepoint.com/_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Strict-Transport-Security: max-age=31536000
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"North Central US","Slice":"SliceC","Ring":"3","ScaleUnit":"003","RoleInstance":"AGSFE_IN_30"}}
Content-Length: 0
```
We then do a GET request on the redirected URL which doesn't work.
```
2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc0004be400)
2020/09/12 22:44:36 DEBUG : GET /_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: redacted-my.sharepoint.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Referer: https://graph.microsoft.com/v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Accept-Encoding: gzip
```
### What did you expect to see?
I expected http.Client to redirect with a DELETE request.
According to [RFC7231](https://tools.ietf.org/html/rfc7231#section-6.4.3)
> 6.4.3. 302 Found
>
> The 302 (Found) status code indicates that the target resource
> resides temporarily under a different URI. Since the redirection
> might be altered on occasion, the client ought to continue to use the
> effective request URI for future requests.
>
> The server SHOULD generate a Location header field in the response
> containing a URI reference for the different URI. The user agent MAY
> use the Location field value for automatic redirection. The server's
> response payload usually contains a short hypertext note with a
> hyperlink to the different URI(s).
>
> Note: For historical reasons, a user agent MAY change the request
> method from POST to GET for the subsequent request. If this
> behavior is undesired, the 307 (Temporary Redirect) status code
> can be used instead.
I note that this only talks about POST to GET and not other methods in the historical reasons section.
So I'm not sure Go's implementation is RFC compliant here - the RFC doesn't say that user agents may change any other sort of method from POST to GET.
The code is here:
https://github.com/golang/go/blob/66e66e71132034aa620ffbae9008f951da0f9f27/src/net/http/client.go#L496-L512
Which says for all methods other than GET or HEAD, change them to GET, whereas the RFC says you can change POST to GET only.
So I think backwards compatibility might demand that POST becomes GET so something like this would be more RFC compliant.
```go
if reqMethod == "POST" {
redirectMethod = "GET"
}
```
The current code seems to think the major distinction between 301/302 and 307/308 is whether the body is resent. I'm not sure the RFC supports that, but using that distinction I think the code code could look like
```go
if ireq.outgoingLength() != 0 {
redirectMethod = "GET"
}
```
Reading the RFC it says nothing about request bodies vs headers. So I think to be most useful and most RFC compliant the 301,302 redirects should work just like the 307,308 redirects now we have the capability to resend the body if needed.
See: https://github.com/golang/go/issues/18570
See: https://forum.rclone.org/t/purge-command-fails-on-onedrive/19048
Ideally Microsoft should be issuing a 307 redirect here which means don't change the HTTP verb. I think Go's behaviour in changing the DELETE is wrong though.
You can do this with rclone by using the --disable Purge flag which stops the special purge method for onedrive running and rclone will revert back to iterating the directory and deleting things individually.
It is possible to make a workaround for this if you are desperate - let me know!