[improvement] List of files with errors

WHen you get an error during sync/copy, there is not no way to know which files are in errors. Looking at the logs with the “-v” error is pain full.
This improvement is to add at the end of the logs a summary of the files that were in error and that were not copied.


Another option (which I would prefer) would be to simply add a convenient, unambiguous “marker” string to each line listing a file in error, like “==Error== <path/filename>”, then it would be a simple matter of grep’ping the output for that string.

What rclone could print at the end is a numeric summary with the number of files with errors, like for example “md5sum -c” does.



Note that rclone will write errors to stderr but the logs from -v to stdout

so if you do

rclone copy -v ..... 2>errors.log >verbose.log

You should be able to see the errors separately

Does that help?.

That doesn’t work that well. Depending on the type of error, the error is in different files.
I use rclone -v … 2>Desktop/rclone_errors.txt >Desktop/rclone_verbose.txt
In rclone_errors.txt, I get errors like:

Failed to copy: invalidRequest: Bad Argument

in rclone_verbose.txt, I can see errors like:

2016/11/04 09:16:51 Couldn’t decode error response: EOF
2016/11/04 09:16:51 pacer: Rate limited, increasing sleep to 20ms
2016/11/04 09:16:51 pacer: low level retry 1/10 (error 502 Bad Gateway: )
2016/11/04 09:16:57 pacer: Reducing sleep to 15ms

So errors can be seen in the 2 files, it’s not easy to understand where the issue is.

I’ve resorted to something like this setup to prune things I don’t care about.

LOGFILTER="eading|Sizes identical|Unchanged skipping|time the same|Excluding from sync|transfer non file/directory|Excluded from sync|Size and MD5 of src and dst objects identical"
CRYPTLOGFILTER="$LOGFILTER|File found with same name but different case"

 rclone copy -v "/data/Media1" "robacd-crypt:/" 2>&1 | egrep -v "$CRYPTLOGFILTER"

This filters things that I consider ‘standard out’ or normal stuff so I can look at errors or odd things.

I think ultimately, rclone would do better if we could specify a loglevel for the output like 'WARN|ERROR|CRTICAL|FATAL" etc. Then we can just select the verbosity level but that would be some future enhancement? :slight_smile:

Bear in mind that rclone does lots of retries of things - you don’t need to see those errors as they will be corrected by a successful action later - those are in the verbose log.

In the error log are the actual errors which couldn’t be corrected.

That is probably a good idea..

Can you add any ideas to this issue please?