What is the problem you are having with rclone?
When I'm trying to upload a large file (a few GB) to my Nextcloud account, I'm getting this error at the end:
Failed to copy: finalize chunked upload failed, destinationURL: "(removed)": merging the uploaded chunks failed with 423 LOCKED. This usually happens when the chunks merging is still in progress on NextCloud, but it may also indicate a failed transfer: "(removed).upload.part" is locked: OCA\DAV\Connector\Sabre\Exception\FileLocked: 423 Locked
But if I wait 7 more minutes, the file appears in my Nexcloud account, and looks fine.
I have seen this topic WebDAV: rclone copy runs into locked files which looks like the same problem, but the suggestions given there didn't work for me. I have increased both timeout and contimeout to 2 hours, and the issue still happens. The total transfer time is less than 2 hours, so it doesn't seem like the timeout settings even apply to that. I couldn't find any setting for Nextcloud chunk merging timeout.
Smaller files, perhaps up to a few hundred MBs, are copied without issues.
Run the command 'rclone version' and share the full output of the command.
rclone v1.63.1
- os/version: Microsoft Windows 10 Home 22H2 (64 bit)
- os/kernel: 10.0.19045.3324 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.20.6
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
WebDAV with Nextcloud
The command you were trying to run (eg rclone copy /tmp remote:tmp)
C:\PProg\Dysk\rclone\rclone.exe copy D:\BackupScript\rclone\test_files nxtcld_crypt:Test `
--config=D:\BackupScript\rclone\rclone.lapasus.nextcloud.config `
--transfers=4 `
--contimeout=2h0m0s `
--timeout=2h0m0s `
--stats=30s `
--retries-sleep=10s `
--retries=1 `
-vv
The rclone config contents with secrets removed.
[nxtcld]
type = webdav
url = https://(removed)/remote.php/dav/files/(removed)/
vendor = nextcloud
user = (removed)
pass = (removed)
[nxtcld_crypt]
type = crypt
remote = nxtcld:Backup/Lapasus
directory_name_encryption = true
password = (removed)
A log from the command with the -vv flag
2023/08/16 17:23:52 DEBUG : rclone: Version "v1.63.1" starting with parameters ["C:\\PProg\\Dysk\\rclone\\rclone.exe" "copy" "D:\\BackupScript\\rclone\\test_files" "nxtcld_crypt:Test" "--config=D:\\BackupScript\\rclone\\rclone.lapasus.nextcloud.config" "--transfers=4" "--contimeout=2h0m0s" "--timeout=2h0m0s" "--stats=30s" "--retries-sleep=10s" "--retries=1" "-vv"]
2023/08/16 17:23:52 DEBUG : Creating backend with remote "D:\\BackupScript\\rclone\\test_files"
Enter configuration password:
password:
2023/08/16 17:23:58 DEBUG : Using config file from "D:\\BackupScript\\rclone\\rclone.lapasus.nextcloud.config"
2023/08/16 17:23:58 DEBUG : fs cache: renaming cache item "D:\\BackupScript\\rclone\\test_files" to be canonical "//?/D:/BackupScript/rclone/test_files"
2023/08/16 17:23:58 DEBUG : Creating backend with remote "nxtcld_crypt:Test"
2023/08/16 17:23:58 DEBUG : Creating backend with remote "nxtcld:Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o"
2023/08/16 17:23:58 DEBUG : found headers:
2023/08/16 17:23:58 NOTICE: Chunks temporary upload directory: https://(removed)/remote.php/dav/uploads/(removed)/
2023/08/16 17:24:05 DEBUG : openSUSE-12.2-DVD-i586.iso: Need to transfer - File not found at Destination
2023/08/16 17:24:05 DEBUG : BMS_connection.png: Size and modification time the same (differ by -950.6405ms, within tolerance 1s)
2023/08/16 17:24:05 DEBUG : BMS_connection.png: Unchanged skipping
2023/08/16 17:24:05 DEBUG : Ogniwa.ods: Size and modification time the same (differ by -896.9225ms, within tolerance 1s)
2023/08/16 17:24:05 DEBUG : Ogniwa.ods: Unchanged skipping
2023/08/16 17:24:05 DEBUG : info.txt: Size and modification time the same (differ by -267.817ms, within tolerance 1s)
2023/08/16 17:24:05 DEBUG : info.txt: Unchanged skipping
2023/08/16 17:24:05 DEBUG : tsdz2-programming-cable-pinout-1.jpg: Size and modification time the same (differ by -740.3863ms, within tolerance 1s)
2023/08/16 17:24:05 DEBUG : tsdz2-programming-cable-pinout-1.jpg: Unchanged skipping
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/EN-Manual_display operation-TSDZ2-mb.20beta1.pdf: Size and modification time the same (differ by 0s, within tolerance 1s)
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/EN-Manual_display operation-TSDZ2-mb.20beta1.pdf: Unchanged skipping
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/instrukcja oryginalna VLCD5.pdf: Size and modification time the same (differ by -617.6553ms, within tolerance 1s)
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/instrukcja oryginalna VLCD5.pdf: Unchanged skipping
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/wgrane ustawienia.txt: Size and modification time the same (differ by -921.2686ms, within tolerance 1s)
2023/08/16 17:24:08 DEBUG : tsdz2-instrukcje/wgrane ustawienia.txt: Unchanged skipping
2023/08/16 17:24:08 DEBUG : openSUSE-12.2-DVD-i586.iso: Update will use the chunked upload strategy
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_option.hex: Size and modification time the same (differ by -523.5018ms, within tolerance 1s)
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_option.hex: Unchanged skipping
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_data.hex: Size and modification time the same (differ by -72.7418ms, within tolerance 1s)
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_data.hex: Unchanged skipping
2023/08/16 17:24:13 DEBUG : Encrypted drive 'nxtcld_crypt:Test': Waiting for checks to finish
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_program.hex: Size and modification time the same (differ by -257.7175ms, within tolerance 1s)
2023/08/16 17:24:13 DEBUG : tsdz2-instrukcje/Org_Software_tsdz2/tsdz2_org_fw_program.hex: Unchanged skipping
2023/08/16 17:24:13 DEBUG : Encrypted drive 'nxtcld_crypt:Test': Waiting for transfers to finish
2023/08/16 17:24:32 INFO :
Transferred: 30 MiB / 4.143 GiB, 1%, 1.228 MiB/s, ETA 57m10s
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso: 0% /4.143Gi, 1.228Mi/s, 57m10s
System.Management.Automation.RemoteException
2023/08/16 17:25:02 INFO :
Transferred: 80 MiB / 4.143 GiB, 2%, 1.258 MiB/s, ETA 55m8s
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 1m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso: 1% /4.143Gi, 1.258Mi/s, 55m8s
System.Management.Automation.RemoteException
2023/08/16 17:25:32 INFO :
Transferred: 120 MiB / 4.143 GiB, 3%, 1.415 MiB/s, ETA 48m32s
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 1m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso: 2% /4.143Gi, 1.415Mi/s, 48m32s
System.Management.Automation.RemoteException
(... removed progress reports ...)
2023/08/16 18:16:32 INFO :
Transferred: 4.117 GiB / 4.143 GiB, 99%, 1.063 MiB/s, ETA 24s
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 52m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso: 99% /4.143Gi, 1.063Mi/s, 24s
System.Management.Automation.RemoteException
2023/08/16 18:17:00 DEBUG : openSUSE-12.2-DVD-i586.iso: Computing sha1 hash of encrypted source
2023/08/16 18:17:02 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 850.125 KiB/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 53m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 850.125Ki/s, -
System.Management.Automation.RemoteException
2023/08/16 18:17:32 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 122.637 KiB/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 53m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 122.637Ki/s, -
System.Management.Automation.RemoteException
2023/08/16 18:18:02 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 17.690 KiB/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 54m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 17.690Ki/s, -
System.Management.Automation.RemoteException
2023/08/16 18:18:32 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 2.552 KiB/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 54m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 2.552Ki/s, -
System.Management.Automation.RemoteException
2023/08/16 18:19:02 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 376 B/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 55m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 376/s, -
System.Management.Automation.RemoteException
2023/08/16 18:19:32 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 54 B/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 55m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 54/s, -
System.Management.Automation.RemoteException
2023/08/16 18:20:02 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 7 B/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 56m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 7/s, -
System.Management.Automation.RemoteException
2023/08/16 18:20:32 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 1 B/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 56m39.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 1/s, -
System.Management.Automation.RemoteException
2023/08/16 18:21:02 INFO :
Transferred: 4.144 GiB / 4.143 GiB, 100%, 0 B/s, ETA -
Checks: 10 / 10, 100%
Transferred: 0 / 1, 0%
Elapsed time: 57m9.8s
Transferring:
* openSUSE-12.2-DVD-i586.iso:100% /4.143Gi, 0/s, -
System.Management.Automation.RemoteException
2023/08/16 18:21:10 DEBUG : pacer: low level retry 1/10 (error <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx</center>
</body>
</html>: 504 Gateway Time-out)
2023/08/16 18:21:10 DEBUG : pacer: Rate limited, increasing sleep to 20ms
2023/08/16 18:21:13 DEBUG : pacer: Reducing sleep to 15ms
2023/08/16 18:21:13 ERROR : openSUSE-12.2-DVD-i586.iso: Failed to copy: finalize chunked upload failed, destinationURL: "https://(removed)/remote.php/dav/files/(removed)/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g": merging the uploaded chunks failed with 423 LOCKED. This usually happens when the chunks merging is still in progress on NextCloud, but it may also indicate a failed transfer: "(removed)/files/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g.upload.part" is locked: OCA\DAV\Connector\Sabre\Exception\FileLocked: 423 Locked
2023/08/16 18:21:13 ERROR : Attempt 1/1 failed with 1 errors and: finalize chunked upload failed, destinationURL: "https://(removed)/remote.php/dav/files/(removed)/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g": merging the uploaded chunks failed with 423 LOCKED. This usually happens when the chunks merging is still in progress on NextCloud, but it may also indicate a failed transfer: "(removed)/files/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g.upload.part" is locked: OCA\DAV\Connector\Sabre\Exception\FileLocked: 423 Locked
2023/08/16 18:21:23 INFO :
Transferred: 4.144 GiB / 4.144 GiB, 100%, 0 B/s, ETA 0s
Errors: 1 (retrying may help)
Checks: 10 / 10, 100%
Elapsed time: 57m30.8s
System.Management.Automation.RemoteException
2023/08/16 18:21:23 DEBUG : 5 go routines active
2023/08/16 18:21:23 Failed to copy: finalize chunked upload failed, destinationURL: "https://(removed)/remote.php/dav/files/(removed)/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g": merging the uploaded chunks failed with 423 LOCKED. This usually happens when the chunks merging is still in progress on NextCloud, but it may also indicate a failed transfer: "(removed)/files/Backup/Lapasus/1m7h4h1ucd0d1na1ogkhojqg1o/hfp6jgvlc76qsiev8h7dhk7ohuuqcoiqfns6olepjn2ju1j2da6g.upload.part" is locked: OCA\DAV\Connector\Sabre\Exception\FileLocked: 423 Locked