OpenDrive and >10gb files - Error message invalid temp file location

What is the problem you are having with rclone?

errors when uploading large files >10gb to opendrive, the error that appears is Invalid Temp File Location

What is your rclone version (output from rclone version)

1.54.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

server 2019 64bit

Which cloud storage system are you using? (eg Google Drive)

opendrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

"C:\rclone\rclone.exe" copy //store03/Rob opendrive_crypt:/test1 --progress --verbose --exclude /@Recently-Snapshot/ --exclude /@Recycle/ --transfers 1 -vv

The rclone config contents with secrets removed.

Paste config here

A log from the command with the -vv flag

![opendrive file size issue Screenshot 2021-03-10 175232|690x161](upload://hHyPTZXTeMyYJWzbMZ0aEANT6Md.png) 

hello and welcome to the forum,

not able to see that screenshot.

if possible, post the actual text of the debug output, not hard to read screenshots.

also, post the config file, redact secrets.

[opendrive]
type = opendrive
username = USERNAME_REMOVED
password = PASSWORD_REMOVED

[opendrive_crypt]
type = crypt
remote = opendrive:/backup
filename_encryption = standard
directory_name_encryption = true
password = PASSWORD_REMOVED
password2 = PASSWORD_REMOVED

[hubic_crypt]
type = crypt
remote = hubic:/default/backup
filename_encryption = standard
directory_name_encryption = true
password = PASSWORD_REMOVED
password2 = PASSWORD_REMOVED

can you post the text, not screenshots. just copy and paste.
enclose the text with three backticks before and after thext.
include the top part of the debug log, that is missing from that snippet.
will look like

2021/03/09 15:23:01 DEBUG : rclone: Version "v1.54.0" starting with parameters ["C:\\data\\rclone\\scripts\\rclone.exe" "sync" "b:\\mount\\rcloner\\data_wasabi_sync+check_20210309.152259\\data\\" "wasabicrypten07data:en07data/data/rclone/backup" "--stats=0" "--fast-list" "--exclude-from=c:\\data\\rclone\\scripts\\rr\\data\\exclude.txt" "--backup-dir=wasabicrypten07data:en07data/data/rclone/archive/20210309.152259" "--log-level=DEBUG" "--log-file=C:\\data\\rclone\\logs\\data_wasabi_sync+check\\20210309.152259\\rclone.log"]
2021/03/09 15:23:01 DEBUG : Creating backend with remote "b:\\mount\\rcloner\\data_wasabi_sync+check_20210309.152259\\data\\"
2021/03/09 15:23:01 DEBUG : Using RCLONE_CONFIG_PASS password.
2021/03/09 15:23:01 DEBUG : Using config file from "C:\\data\\rclone\\scripts\\rclone.conf"
2021/03/09 15:23:01 DEBUG : fs cache: renaming cache item "b:\\mount\\rcloner\\data_wasabi_sync+check_20210309.152259\\data\\" to be canonical "//?/b:/mount/rcloner/data_wasabi_sync+check_20210309.152259/data/"
2021/03/09 15:23:01 DEBUG : Creating backend with remote "wasabicrypten07data:en07data/data/rclone/backup"
2021/03/09 15:23:01 DEBUG : Creating backend with remote "wasabi01:en07data/u1538dp9419lbg721ut4n6r4i0/58faqsuggu22ctl3o44q21e0ug/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg"
2021/03/09 15:23:01 DEBUG : Creating backend with remote "wasabicrypten07data:en07data/data/rclone/archive/20210309.152259"
2021/03/09 15:23:01 DEBUG : Creating backend with remote "wasabi01:en07data/u1538dp9419lbg721ut4n6r4i0/58faqsuggu22ctl3o44q21e0ug/vplun6pmri0mr96fadebs0mm0s/liigm7fdonh9ek3p5g2bvri3q0/j7imulatcegmtdo8flbfdnrfhs"

sorry i didnt have the log file, i guess i need to lookup how to get this into a text file. It took hours to upload to get it to fail, everytime ive checked it i get the same error about "invalid temp file location", no matter what files i upload if they are over 10gb. There is plenty of disk space so wasnt sure where this temp location was

try copying just that one file.
i do not use opendrive, but the more info you can post, the better chance an opendrive expert can help you.

no problem i have a running, ill leave it run over night and hopefully we will have something more useful to work with tomorrow. thanks for prompt response

ok ive got the log file but cant upload it because im a new user?

ok got the log file and its now attached, if there is anything further please let me know.

rclonelog.txt (450.8 KB)

not sure what that error is.

I'll just note that that is a message from the OpenDrive server so contacting support might help.

ok thanks, im jsut awaiting a reply from their developer team

ok ive managed to get a reply from their development team,

The error occurred due to the fact that the client application returned a timeout error without waiting for a response from the server. But close_file_upload processing continues on the server. When the request is repeated, code 423 is returned with the message "This file is already being processed. Please wait until previous upload operation will be finished."

close_file_upload can take a very long time for large files. The solution is to increase the timeout to 20 minutes if the client application allows it.

You can control rclone's timeout with the --timeout flag. The default is 5m for 5 minutes.

Can you have an experiment with that to see if that helps?

thanks for this, ive set it to "--timeout 15m" and the test file i have been using has successfully uploaded. im going to attempt experiment further (and will provide further feedback) but thought it would be useful at least confirming a test that was always failing as now succeeding incase others are experiencing similar issues.

Great - that is useful.

Maybe the onedrive backend should adjust the timeout here (might be hard).

At very minimum we should put a note in the docs about it.

I'm interested to hear your further testing :slight_smile:

So just to confirm that setting the time out to 15 minutes seemed to give some good results, when i saw one or two failures i increased it to 30 minutes I want to guess it may have improved things slightly. I dont claim to fully understand how this works from a technical point of view only that it seems to appear to fix this for larger files with OpenDrive.

Prior to doing any of these changes I also tried uploading large files direct to OpenDrive and they uploaded successfully, so its definately linked to these timeouts.

Hopefully this information helps anyone else experiencing similar issues :slight_smile:

1 Like

Good.

Do you fancy submitting a patch to the onedrive docs? Click the pencil icon here and I suggest a mention in the Limitations section: rclone/onedrive.md at master · rclone/rclone · GitHub

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.