After make it works with fastweb[1] a problem arise (it happened also with curl)
If upload time is higher than 1 minute you get a 504 Gateway Time-out
Fact is file is uploaded correctly.
Is there a way to ignore this errore and have rclone doing is stuff.
Hi I build from source with some minor edit to webdav.go
// retryErrorCodes is a slice of error codes that we will retry
156 var retryErrorCodes = []int{
157 429, // Too Many Requests.
158 500, // Internal Server Error
159 502, // Bad Gateway
160 503, // Service Unavailable
161 //504, // Gateway Timeout
162 509, // Bandwidth Limit Exceeded
163 }
164
165 // shouldRetry returns a boolean as to whether this resp and err
166 // deserve to be retried. It returns the err as a convenience
167 func shouldRetry(resp *http.Response, err error) (bool, error) {
168 if resp.StatusCode == 504 {
169 err = nil
170 }
171 return fserrors.ShouldRetry(err) || fserrors.ShouldRetryHTTP(resp, retryErrorCodes), err
172 }
and after a build:
$./rclone --low-level-retries 1 --retries 1 -vv copy -P FILE.mkv sec:Anime/
2019/01/10 15:07:07 DEBUG : rclone: Version "v1.45-DEV" starting with parameters ["./rclone" "--low-level-retries" "1" "--retries" "1" "-vv" "copy" "-P" "FILE.mkv" "sec:Anime/"]
2019/01/10 15:07:07 DEBUG : Using config file from "/home/ME/.config/rclone/rclone.conf"
2019-01-10 15:07:08 DEBUG : FILE.mkv: Couldn't find file - need to transfer
2019-01-10 15:19:35 INFO : FILE.mkv: Copied (new)
Transferred: 7.424G / 7.424 GBytes, 100%, 10.166 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 12m27.7s
2019/01/10 15:19:35 INFO :
Transferred: 7.424G / 7.424 GBytes, 100%, 10.166 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 12m27.7s
2019/01/10 15:19:35 DEBUG : 4 go routines active
2019/01/10 15:19:35 DEBUG : rclone: Version "v1.45-DEV" finishing with parameters ["./rclone" "--low-level-retries" "1" "--retries" "1" "-vv" "copy" "-P" "FILE.mkv" "sec:Anime/"]
Now I have same questions:
It did works? Is there a way to perform a check without downloading the remote file and md5sum both files?
What do you think about a --ignore-http-statuscode list_of_status_code_comma_separated
flag?
Ok, I think you can implement something like this:
In every backend where shouldRetry and retryErrorCodes is defined
you should replace shouldRetry with this:
func shouldRetry(resp *http.Response, err error) (bool, error) {
if resp != nil {
for _, i := range IgnoreStatusCode {
if resp.StatusCode == i {
return false, nil
}
}
}
return fserrors.ShouldRetry(err) || fserrors.ShouldRetryHTTP(resp, retryErrorCodes), err
}
Now we need to populate IgnoreStatusCode from the command line --ignore-http-status-code 504,505
Then if I understood cobra a little (I didn’t). you should add somewhere: RootCmd.PersistentFlags().String("ignorehttp","","Ignore http status code and keep going")
and then
IgnoreHttpStatusCode := cmd.Flag("ignorehttp").Value.String()
tmp := strings.Split(IgnoreHttpStatusCode, ",")
var IgnoreStatusCode []int{}
for _, i in range(tmp) {
statuscode, err := strconv.Atoi(i)
if err != nil {
MEGAERROR_STOPEXECUTION()
}
IgnoreStatusCode = append(IgnoreStatusCode ,statuscode)
}
{
Name: "ignore504",
Help: "treat http status 504 as success.",
},
is enough? Onestly I am lost here.
nope, only seen 504 on this server, but because you use retryErrorCodes on other backends I thought maybe a global flag to disable some of them could be useful.
Either way I will try to write something in golang.
I’m not sure, but it seems that after 504 the modtime is not updated with the original modtime.
I’m performing some experiment using moveto (to a Crypt over Webdav) and for large files I’m experiencing this issue. Not sure the root cause.
