Rclone exists with exit code 1, not sure why?

asdffdsa · February 12, 2022, 3:13pm

What is the problem you are having with rclone?

rclone ends with exit code 1. not sure why??

about the rclone debug log,

i have included the tops lines.
i have included the last two lines.
rclone just stop at that point. nothing in the debug log about the reason.
for the complete log, there were no errors, pacer, retries, timeouts, etc.

rclone is orchestrated by a python script, and i have included that log file as overview.log
notice at the end of the log, rclone exists with exit code 1.
not sure why???

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0
- os/version: Microsoft Windows 10 Pro 2009 (64 bit)
- os/kernel: 10.0.22000.434 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.17.2
- go/linking: dynamic
- go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

wasabi, s3 clone

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

rclone.exe sync "b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data" "wasabi_en07-data-crypt:en07data/rclone/backup" --stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt  --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log

The rclone config contents with secrets removed.

[wasabi_en07-data-crypt_remote]
type = s3
provider = Wasabi
access_key_id = redacted
secret_access_key = redacted
endpoint = s3.us-east-2.wasabisys.com

[wasabi_en07-data-crypt]
type = crypt
remote = wasabi_en07-data-crypt_remote:en07.data.crypt
filename_encryption = standard
directory_name_encryption = true
password = redacted
password2 = redacted

A log from the command with the `-vv` flag

DEBUG : rclone: Version "v1.57.0" starting with parameters ["c:\\data\\rclone\\scripts\\rclone.exe" "sync" "b:\\rclone\\vss\\data_files_wasabi_en07-data-crypt_20220212.084210\\data" "wasabi_en07-data-crypt:en07data/rclone/backup" "--stats=0" "--fast-list" "--s3-no-check-bucket=true" "--exclude-from=c:\\data\\rclone\\scripts\\rr\\data\\exclude.txt" "--config=c:\\data\\rclone\\scripts\\rclone.conf" "--ask-password" "--log-level=DEBUG" "--log-file=c:\\data\\rclone\\logs\\data_files_wasabi_en07-data-crypt\\20220212.084210\\rclone.log"]
DEBUG : Creating backend with remote "b:\\rclone\\vss\\data_files_wasabi_en07-data-crypt_20220212.084210\\data"
DEBUG : Using config file from "c:\\data\\rclone\\scripts\\rclone.conf"
DEBUG : fs cache: renaming cache item "b:\\rclone\\vss\\data_files_wasabi_en07-data-crypt_20220212.084210\\data" to be canonical "//?/b:/rclone/vss/data_files_wasabi_en07-data-crypt_20220212.084210/data"
DEBUG : Creating backend with remote "wasabi_en07-data-crypt:en07data/rclone/backup"
DEBUG : Creating backend with remote "wasabi_en07-data-crypt_remote:en07.data.crypt/u1538dp9419lbg721ut4n6r4i0/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg"
DEBUG : wasabi_en07-data-crypt_remote: detected overridden config - adding "{Dn7qA}" suffix to name
DEBUG : fs cache: renaming cache item "wasabi_en07-data-crypt_remote:en07.data.crypt/u1538dp9419lbg721ut4n6r4i0/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg" to be canonical "wasabi_en07-data-crypt_remote{Dn7qA}:en07.data.crypt/u1538dp9419lbg721ut4n6r4i0/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg"
DEBUG : fs cache: switching user supplied name "wasabi_en07-data-crypt_remote:en07.data.crypt/u1538dp9419lbg721ut4n6r4i0/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg" for canonical name "wasabi_en07-data-crypt_remote{Dn7qA}:en07.data.crypt/u1538dp9419lbg721ut4n6r4i0/vplun6pmri0mr96fadebs0mm0s/ebue23o4ems817a5kdmjh5ureg"
...
DEBUG : file.txt: Size and modification time the same (differ by 0s, within tolerance 100ns)
DEBUG : file.txt: Unchanged skipping

overview.log

database.ini
TheDateTime=20220212.084210
BackupNameFull=data_files_wasabi_en07-data-crypt
BackupName=data
Start_VSS__init__
VSS.BackupNameFull=data_files_wasabi_en07-data-crypt
VSS.LogDir=c:\data\rclone\logs\vs
Start MakeDirs - c:\data\rclone\logs\vs
VSSMountPoint=b:\rclone\vss
VSS.VSSMountPoint=b:\rclone\vss
End_VSS__init__
Start - DoVSSDeleteAll
Delete VSSMountPoints
DoVSSDeleteAll vssel - c:\data\rclone\logs\vs\data_files_wasabi_en07-data-crypt_20220212.084024-{46e69d92-a76a-4329-9343-504d6988ed95}.vshadowmountdir.log
DoVSSDeleteAll - listfiles - vsslogname=data_files_wasabi_en07-data-crypt_20220212.084024
Start RemoveItem - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084024
Removing Item - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084024
End RemoveItem
Start RemoveItem - c:\data\rclone\logs\vs\data_files_wasabi_en07-data-crypt_20220212.084024-{46e69d92-a76a-4329-9343-504d6988ed95}.vshadowmountdir.log
Removing Item - c:\data\rclone\logs\vs\data_files_wasabi_en07-data-crypt_20220212.084024-{46e69d92-a76a-4329-9343-504d6988ed95}.vshadowmountdir.log
End RemoveItem
Start DoCmd DoVSSDeleteAll-queryall=cmd /c c:\data\rclone\scripts\vshadow.exe -q > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vsqueryall.txt
DoCmd: End:   DoVSSDeleteAll-queryall=cmd /c c:\data\rclone\scripts\vshadow.exe -q > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vsqueryall.txt
DoVSSDeleteAll-List files in c:\data\rclone\logs\vs\{46e69d92-a76a-4329-9343-504d6988ed95}-data_files_wasabi_en07-data-crypt.vshadow.log
DoVSSDeleteAll - listfiles - SnapshotID={46e69d92-a76a-4329-9343-504d6988ed95}
DoVSSDeleteAll - listfiles - {46e69d92-a76a-4329-9343-504d6988ed95} in c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vsqueryall.txt
Start DoCmd DoVSSDeleteAll-VssDirFound vshadow.exe -ds={46e69d92-a76a-4329-9343-504d6988ed95}=cmd /c c:\data\rclone\scripts\vshadow.exe -ds={46e69d92-a76a-4329-9343-504d6988ed95}
DoCmd: End:   DoVSSDeleteAll-VssDirFound vshadow.exe -ds={46e69d92-a76a-4329-9343-504d6988ed95}=cmd /c c:\data\rclone\scripts\vshadow.exe -ds={46e69d92-a76a-4329-9343-504d6988ed95}
Start RemoveItem - c:\data\rclone\logs\vs\{46e69d92-a76a-4329-9343-504d6988ed95}-data_files_wasabi_en07-data-crypt.vshadow.log
Removing Item - c:\data\rclone\logs\vs\{46e69d92-a76a-4329-9343-504d6988ed95}-data_files_wasabi_en07-data-crypt.vshadow.log
End RemoveItem
End - DoVSSDeleteAll
Start ProcessBackup
SourceDir=c:\data
DoVShadow=True
VSSMountPoint=b:\rclone\vss
DoRcloneSyncFiles=True|--stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt
DoRcloneCryptCheckFiles=True|--stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt
RcloneLogLevel=--log-level=DEBUG
DestDirRclone=wasabi_en07-data-crypt:en07data
RcloneConfigFile=rclone.conf
DoProcessLogs=True
DoSendEmailLogs=True
PathType=local
VShadowMountDir=b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
SourceDir=b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data
DestDirRclone=wasabi_en07-data-crypt:en07data
PathType=remote
DestDirRclonePathType=remote
RcloneGlobalFlags= --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log
RcloneSyncFilesCmd=c:\data\rclone\scripts\rclone.exe sync "b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data" "wasabi_en07-data-crypt:en07data/rclone/backup" --stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt  --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log
RcloneCryptCheckFilesCmd=c:\data\rclone\scripts\rclone.exe cryptcheck "b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data" "wasabi_en07-data-crypt:en07data/rclone/backup" --stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt  --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log
Start - DoVSSDeleteAll
Delete VSSMountPoints
Start DoCmd DoVSSDeleteAll-queryall=cmd /c c:\data\rclone\scripts\vshadow.exe -q > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vsqueryall.txt
DoCmd: End:   DoVSSDeleteAll-queryall=cmd /c c:\data\rclone\scripts\vshadow.exe -q > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vsqueryall.txt
End - DoVSSDeleteAll
Start_VSS__Create
SourceDir=c:\data
SourceDirRelative=data
SourceDriveToBeShadowed=c:
Start DoCmd vss-create=cmd /c c:\data\rclone\scripts\vshadow.exe -p -nw c: > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vshadow.log
DoCmd: End:   vss-create=cmd /c c:\data\rclone\scripts\vshadow.exe -p -nw c: > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vshadow.log
VSS.Create SnapshotID={cb9e708d-f455-4025-8555-2d652fb216a4}
VShadowMountDir=b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
Start RemoveItem - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
Item Does Not Exist - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
End RemoveItem
Start MakeDirs - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
Making Dir - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210
End MakeDirs
Start DoCmd vshasdow.el=cmd /c c:\data\rclone\scripts\vshadow.exe -el={cb9e708d-f455-4025-8555-2d652fb216a4},b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210 > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vs.el.txt
DoCmd: End:   vshasdow.el=cmd /c c:\data\rclone\scripts\vshadow.exe -el={cb9e708d-f455-4025-8555-2d652fb216a4},b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210 > c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\vs.el.txt
VSS.Create SnapshotID={cb9e708d-f455-4025-8555-2d652fb216a4}
SourceDir=b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data
End_VSS__Create__
SourceDir exist - b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data
Start DoCmd RcloneSyncFiles=c:\data\rclone\scripts\rclone.exe sync "b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data" "wasabi_en07-data-crypt:en07data/rclone/backup" --stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt  --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log
DoEnd=DoCmd: End ERROR: RcloneSyncFiles=c:\data\rclone\scripts\rclone.exe sync "b:\rclone\vss\data_files_wasabi_en07-data-crypt_20220212.084210\data" "wasabi_en07-data-crypt:en07data/rclone/backup" --stats=0 --fast-list --s3-no-check-bucket=true --exclude-from=c:\data\rclone\scripts\rr\data\exclude.txt  --config=c:\data\rclone\scripts\rclone.conf "--ask-password" --log-level=DEBUG --log-file=c:\data\rclone\logs\data_files_wasabi_en07-data-crypt\20220212.084210\rclone.log with returncode=1

ncw · February 14, 2022, 3:05pm

I would expect to see a log with ERROR if rclone exits with a non zero return code.

Turn the stats back on --stats 1h and you'll get a final log with the number of errors rclone thinks it had

asdffdsa · February 16, 2022, 12:40am

thanks, the good thing; this was a one time occurrence.

tho some of my backups fail now and again, cannot remember a case like this, nothing in the debug log.

but in general, your suggestion is good so i changed 50+ backup configs to use --stats=10h, not --stats=1h
hopefully this will never happen again, but it it does, i will have that debug log.

ncw · February 16, 2022, 11:52am

Cross fingers!

Its a bug if rclone exists with non zero error without writing an ERROR log (once it has got going anyway), so if we can track it down, we can fix it!

system · March 18, 2022, 11:53am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.